VC++ - Arduino program stops midway while executing

Projects General Guidance
1

I hope I'm posting this in the right forum (it is my first time posting in this forum).

I am interfacing my vision program (written in openCV - so in C++) with my arduino through the serial port. The main idea of the c++ code is this: it detects an orange ball, and depending on where in the screen it is, it sends a message to the arduino to move the body of the robot to approximately 10 cm of the ball and stop. So depending on where the ball is, I am sending the arduino a character over the serial, and am then waiting for a reply from the arduino before doing anything else with the c++ code (i.e. with while(arduino_reply == NULL){} (an empty while loop)). The arduino's code sends a character (the same character) to the c++ program every time it finishes an activity --- for instance after moving the robot forward, or left, right, etc.

Here's the c++ code:

#include "opencv2/objdetect/objdetect.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"

#include <iostream>
#include "tserial.cpp"
#define DETECTION_BOX 60

using namespace cv;
using namespace std;

//SERIAL COM BEGIN  
Tserial *arduino_com;
//SERIAL COM END

void show(const char* windowName, Mat imageName)
{
  namedWindow(windowName);
  imshow(windowName, imageName); 
}

int main(int argc, char** argv)
{ 
  arduino_com = new Tserial();
  if(arduino_com != 0)
  {
	  cout << "Successfully connected to arduino." << endl;
	  arduino_com->connect("COM5", 19200, spNONE); //57600
  }

  VideoCapture webcamImg(0);
  if(webcamImg.isOpened() == false) 
  {
    std::cout << "error: webcam not accessed successfully\n\n"; 
    return -1;   
  }
  
  char usrInput=0;
  
  Mat img, hsv;
  Mat dilated, eroded;
  Mat afterInRange;
  Mat afterGaussBlur;
  
  vector<Vec3f> circles;
  int minRad = 5;
  int maxRad = 200;
  
  int radius=0;
  
  //char ardCommunication = 'L';

  webcamImg >> img;

  while(usrInput != 27)
  { 

    dilate(img,dilated,Mat());
    erode(dilated,eroded,Mat());
    
    GaussianBlur(eroded, afterGaussBlur, Size(9,9),0,0);
    cvtColor(afterGaussBlur, hsv, CV_BGR2HSV);
    inRange(hsv, Scalar(6,77,199), Scalar(25,250,255), afterInRange);
    
    
    HoughCircles(afterInRange, circles, CV_HOUGH_GRADIENT, 2, afterInRange.rows/2, 50, 25, minRad, maxRad);
    


	// RELEVANT PART
	if(circles.size() == 0)
	{
		arduino_com->sendChar('N');
		cout << "sent N" << endl; 
	}
       // RELEVANT PART END



	//SHOW CENTER OF SCREEN BOX AND POINT
	Point centerOfScreen(320,240);
	Point c1(centerOfScreen.x - DETECTION_BOX, centerOfScreen.y - DETECTION_BOX);
	Point c2(centerOfScreen.x + DETECTION_BOX, centerOfScreen.y + DETECTION_BOX);
	rectangle(img, c1, c2, Scalar(0,255,255), 2,8,0);
	circle(img, centerOfScreen, 2, Scalar(255,0,0), -1, 8, 0);
	int pixelDiameter;

    for(int i=0; i < circles.size(); i++)
    {
      Point centerOfCircle(cvRound(circles[i][0]), cvRound(circles[i][1]));
      radius = cvRound(circles[i][2]);
	  pixelDiameter = radius*2;
      //circle(img, centerOfCircle, radius, Scalar(255,0,0), 2, 8, 0);
	  circle(img, centerOfCircle, 2, Scalar(0,0,255), -1, 8, 0); //SHOWS CENTER OF RECTANGLE
      Point pt1(centerOfCircle.x - radius - 3, centerOfCircle.y + radius + 3);
	  Point pt2(centerOfCircle.x + radius + 3, centerOfCircle.y - radius - 3);
      rectangle(img, pt1, pt2, Scalar(0,255,0), 2, 8, 0);
	  
	  //For a specific circle(rectangle) in the loop - ASSUMING THERE IS ONLY ONE CIRCLE
	  //if x coord circCenter > x coord scrCenter + 80


// ----------RELEVANT PART BEGIN
          if (centerOfCircle.x >= (centerOfScreen.x + DETECTION_BOX))
	  {
		  //Move whole body right
		  //Send arduino 'W'
		  arduino_com->sendChar('W');
		  cout << "sent W" << endl;
	  }
	  else if (centerOfCircle.x <= (centerOfScreen.x - DETECTION_BOX))
	  {
		  //Move whole body left
		  //Send arduino 'A'
		  arduino_com->sendChar('A');
		  cout << "sent A" << endl;
	  }
	  else if (centerOfCircle.y >= (centerOfScreen.y + DETECTION_BOX))
	  {
		  //Tilt camera down
		  //Send arduino 'S'
		  arduino_com->sendChar('S');
		  cout << "sent S" << endl;
	  }
	  else if (centerOfCircle.y <= (centerOfScreen.y - DETECTION_BOX))
	  {
		  //Tilt camera up
		  //Send arduino 'D'
		  arduino_com->sendChar('D');
		  cout << "sent D" << endl;
	  }

	  //If in center
	  else if( (centerOfCircle.x < centerOfScreen.x + DETECTION_BOX) && (centerOfCircle.x > centerOfScreen.x - DETECTION_BOX) && (centerOfCircle.y > centerOfScreen.y - DETECTION_BOX) && (centerOfCircle.y < centerOfScreen.y + DETECTION_BOX) )
	  {
		  arduino_com->sendChar('G');
		  cout << "sent G" << endl;
	  }
    }
    show("orig", img);

	while(arduino_com->getChar() == NULL)
	{}
	
	cout << "Got mesg from Arduino!" << endl;
// ----------RELEVANT PART END

	webcamImg >> img;
	usrInput = waitKey(10);
  }
  
  arduino_com->disconnect();
  delete arduino_com;
  arduino_com = 0;

  return 0;
}

I have highlighted the relevant parts of the code...so that you could ignore the vision parts of it if you are not familiar with the vision aspect of the code.

And here's the arduino code:

//BASE 
#include <SoftwareSerial.h>
#include <Servo.h>
#include <NewPing.h>

int trigPin = 7;
int echoPin = 8;

Servo cameraTilt;
int cameraTiltPin = 9;
int stopVal = 30;

//WHEELS
Servo leftWheel, rightWheel;
int leftWheelPin = 10;
int rightWheelPin = 11; 
int stopServoValue = 94;
int leftSpeed = 25;
int rightSpeed = 25;

int tiltSpeed = 1; 

int robotStopThreshold = 10; //in cm

NewPing sonar(7, 8, 200);
int getDist()
{
  delay(50);                      
  unsigned int uS = sonar.ping(); 
  int cm = sonar.convert_cm(uS);
  return cm;
}

void setup()  
{
  pinMode(13, OUTPUT);
  Serial.begin(19200);
  //mySerial.begin(19200);
  
  cameraTilt.attach(cameraTiltPin);
  leftWheel.attach(leftWheelPin);
  rightWheel.attach(rightWheelPin);
  stopRobot();
  cameraTilt.write(stopVal); 
}

void loop()
{
  while(Serial.available() == 0)
  {
    digitalWrite(13, HIGH);
  }
  
    digitalWrite(13, LOW);
    char value = Serial.read();
    
    if((getDist() > 0) && (getDist() < 10))
    {
      stopRobot();
    }
    else
    {
      if(value == 'G')
      {
        goForward();
        //Serial.write('Y');
      }
      else if(value == 'W')
      {
        //move body right
        goRight();
        //Serial.write('U');
      }
      else if(value == 'A')
      {
        //move body left
        goLeft();
        //Serial.write('I');
      }
      else if(value == 'S')
      {
        //tilt camera down
        if((cameraTilt.read() + tiltSpeed) <= 70)
        {
          cameraTilt.write(cameraTilt.read() + tiltSpeed);
        }
        else
        { 
          cameraTilt.write(70);
        }
        //Serial.write('O');
      }
      else if(value == 'D')
      {
        //tilt camera up
        if((cameraTilt.read() - tiltSpeed) >= 30)
        {
          cameraTilt.write(cameraTilt.read() - tiltSpeed);
        }
        else
        {
          cameraTilt.write(30);
        }
        //Serial.write('P');
      }
      else if(value == 'N')
      {
        //stop robot
        stopRobot();
        //Serial.write('L');
      }
      Serial.print('D');
    } 
}

void goForward()
{
  leftWheel.write(stopServoValue + leftSpeed);
  rightWheel.write(stopServoValue - rightSpeed);
  delay(250);
  stopRobot();
  delay(50);
}

void goLeft()
{
  leftWheel.write(stopServoValue - leftSpeed);
  rightWheel.write(stopServoValue - rightSpeed);
  delay(50);
  stopRobot();
  delay(50);
}

void goRight()
{
  leftWheel.write(stopServoValue + leftSpeed);
  rightWheel.write(stopServoValue + rightSpeed);
  delay(50);
  stopRobot();
  delay(50);
}

void stopRobot()
{
  leftWheel.write(stopServoValue+1);
  rightWheel.write(stopServoValue);
}

Now to my problem. When I run the above C++ program in Visual C++ 2010, the program executes for a few seconds (maybe around 5-10 secs) and then it just freezes...

So the terminal window of the program might look something like this (an example):

Sent G
Got mesg from Arduino!
Sent B
Got mesg from Arduino!
Sent S
Got mesg from Arduino!
Sent G
Got mesg from Arduino!
Sent G
Got mesg from Arduino!
Sent G

<then the program freezes at this point>

Can anyone please guide me through this? I have been troubleshooting this whole project for weeks (not just the basic movement part of the robot, but also other parts like the arm, etc.) and can't think about anything else but this. So it is bothering me a lot. XD

Thank you very much for any help. Much appreciated!

(If you have any further questions...please ask.)

2

Does the Arduino function correctly for a longer time if you send characters to it manually from Serial Monitor or a terminal program instead of the C++ code?

-br

3

Yes it is working exactly the way it should when I'm controlling it from the serial monitor.

4

This may not be your bug, but I wonder if matters that if the first condition is true there is no "D" response sent to the PC.

    if((getDist() > 0) && (getDist() < 10))
    {
      stopRobot();
    }
    else
    {…
        Serial.print('D');
    }

-br

5

I haven't looked at the code and this may not be causing the current problem, but the algorithm you describe would enter a deadlock if either the command or the response was lost or corrupted. It would be safer to have one side apply a timeout and resume working if the other side was unresponsive. In this case, it seems most obvious to have your VC program time out and resume if it fails to get an acknowledgment from the Arduino in a reasonable time.

6

billroy, i tried your suggestion and it did seem to stop the freezing problem...but now the robot is acting quite erratically. another something to debug it seems. i'll work on it again tomorrow i suppose when i regain my sanity. i'll keep you updated on that.

peter, i'll add that to my code too. thank you.

7

Billroy, I tried running the robot on a different surface where there is more contrast between the surface and the ball, and it worked fine that time. I think that that problem is solved. Thank you very much for your help! :grin: