OpenCV and Arduino Serial Communications via USB.

Hi everyone,

I'm currently working on a project which involves computer vision and an Arduino board to control a Stepper motor.

The idea of my project is to search for a small dark spot on a white area using a webcam and then the laser will point out that spot.

I use a webcam and the OpenCV library to detect black spot on a white background. After determining the pixel of that spot, I want to send this data(pixel / x-y position) to the serial port of my laptop and then my Arduino board (Atmega328) connected via USB will access that data and then use it to control 2 servo motors (x and y) which has mirrors mounted on it to reflect the laser.

What I can do now is to search for the spot of a black spot and determine it's pixel(coordinates). I can also control the stepper motor using my Arduino board.

My problem is the communications of my laptop and the Arduino board. How can I send the data to my serial port and then send it to my board.

I really need help. Thanks!

Unless your webcam detects the position of the laser spot and uses positive feedback to drive the laser to the required position, this is going to be hard. If you physically attach the webcam to the laser so that they steer together, you could make it work reasonably well at a fixed distance. That might be all you need.

Otherwise, you need to estimate the position of the marker in 3D space, know the position of the laser in that 3D space and calculate what direction it needs to shine in to intersect the marker. That will involve some moderately complex 3D trig. This is a very similar problem to augmented reality marker tracking and you will be able to find standard libraries to do it, but it will not be easy.

Once you have worked out the answer, sending those steering instructions from the PC to an Arduino and have the Arduino move some stepper motors to the corresponding position is "just" a matter of building the hardware and writing a sketch; the sketch would not be particularly hard if you're familiar with Arduino programming but might be quite challenging for a novice.

I did a similar experiment on one of my robot projects. I used Python+OpenCV to track a golf ball/lighter/any colored object on a webcam stream. The script checked if the object being tracked was on the middle of the picture, and if not it sent commands to the Arduino to rotate the webcam until the object was in the middle again.

Here is a short video about the experiment: http://virtuoosi.org/uploadi/objektin_seurantaa.mpeg

If you don't want the webcam to rotate with the laser you will have to include some inverse dynamics in order to know how to point the laser at a certain location using coordinates.

One option is to have OpenCV track both the black dot and the (red?) dot created by the laser.

Hi guys,

I know that this will be hard. Anyway, can I actually look on your codes? I want to see if how you read-write on serial port. That is my main objective right now.

I'm now able to get the pixel position of my target e.g (positionX,positionY) I use MS Visual Studio to compile. What I want now is to write this data (positionX,positionY) to the serial port so my Arduino board can communicate with.

Any suggestions would be really helpful.

Did you try google? I program in java, so don't know much about .not, but I'm pretty sure there's some serial communication libraries for .net.. (serial communication .net or whatever language you're writing in...)

gilberto: Anyway, can I actually look on your codes? I want to see if how you read-write on serial port. That is my main objective right now.

My code won't help you as I work with Python.

remiss: Did you try google? I program in java, so don't know much about .not, but I'm pretty sure there's some serial communication libraries for .net.. (serial communication .net or whatever language you're writing in...)

As remiss said, Googling for a serial communication library will most likely yield some positive results.