Hi all, I've been trying to figure this out for a couple days now and I've hit a wall with this particular solution (my first time using servos in a dynamic application).
The Halloween project I'm working on is a simple face tracker using 2 servos (pan/tilt), Processing and OpenCV for face tracking but I cannot seem to get the servos to behave.
The goal is just for the servos to track the face position and follow the face.
The original project code (which is now a bit outdated) is outlined here: Face Tracking with a Pan/Tilt Servo Bracket - SparkFun Electronics
I looked everywhere for a more updated version of this without success (there doesn't seem to be a similar project outlined in the last 5 years or so).
The code/libraries have been updated to work with the latest version of Processing 3 and OpenCV 3.4.3 (2018). The PC I'm using is running Windows 7 64-Bit.
I'm fairly certain the problem is with the Arduino code interpreting the serial signals from Processing. I could be wrong but the output from Processing seems to be correct. If you guys think this question is better asked on the Processing forums let me know but I think this is probably an issue with the Arduino code.
Here's a summary:
- The yellow servo signal wires are connected to pins 2 and 3 on the Arduino
- Servos are powered by an external 6V 2amp power supply
- I've pretty much confirmed this is not a hardware issue (see video below), I've tried the same code on a different board with the same results
- In the code, if a face is not detected it spits out a "0", if detected it spits out a "1" and then the related position of the face.
The problem: The servos don't react properly to the face tracking positions, they just keep drifting in a single direction until the end of their travel is reached. Seems like it could be an issue with the math that's above my head. Can anyone identify a fault in the code or maybe something I'm missing?
Thanks very much for taking a look. I made a quick video showing the problem:
ARDUINO CODE:
#include <Servo.h> //Used to control the Pan/Tilt Servos
//These are variables that hold the servo IDs.
char tiltChannel=0, panChannel=1;
//These are the objects for each servo.
Servo servoTilt, servoPan;
//This is a character that will hold data from the Serial port.
char serialChar=0;
void setup(){
servoTilt.attach(2); //The Tilt servo is attached to pin 2.
servoPan.attach(3); //The Pan servo is attached to pin 3.
servoTilt.write(90); //Initially put the servos both
servoPan.write(90); //at 90 degress.
Serial.begin(57600); //Set up a serial connection for 57600 bps.
}
void loop(){
while(Serial.available() <=0); //Wait for a character on the serial port.
serialChar = Serial.read(); //Copy the character from the serial port to the variable
if(serialChar == tiltChannel){ //Check to see if the character is the servo ID for the tilt servo
while(Serial.available() <=0); //Wait for the second command byte from the serial port.
servoTilt.write(Serial.read()); //Set the tilt servo position to the value of the second command byte received on the serial port
}
else if(serialChar == panChannel){ //Check to see if the initial serial character was the servo ID for the pan servo.
while(Serial.available() <= 0); //Wait for the second command byte from the serial port.
servoPan.write(Serial.read()); //Set the pan servo position to the value of the second command byte received from the serial port.
}
//If the character is not the pan or tilt servo ID, it is ignored.
}
PROCESSING CODE:
(Note the included libraries at the top that are fairly easy to install right from the Processing interface)
import gab.opencv.*;
import processing.video.*;
import java.awt.*;
import processing.serial.*; //The serial library is needed to communicate with the Arduino.
Capture video;
OpenCV opencv;
/////////////////////////////////////////////////////////////
//Screen Size Parameters
int width = 640;
int height = 480;
Serial port; // The serial port
//Variables for keeping track of the current servo positions.
char servoTiltPosition = 90;
char servoPanPosition = 90;
//The pan/tilt servo ids for the Arduino serial command interface.
char tiltChannel = 0;
char panChannel = 1;
//These variables hold the x and y location for the middle of the detected face.
int midFaceY=0;
int midFaceX=0;
//The variables correspond to the middle of the screen, and will be compared to the midFace values
int midScreenY = (height/2);
int midScreenX = (width/2);
int midScreenWindow = 10; //This is the acceptable 'error' for the center of the screen.
//The degree of change that will be applied to the servo each time we update the position.
int stepSize=1;
//////////////////////////////////////////////////
void setup() {
String[] cameras = Capture.list();
size(640, 480);
video = new Capture(this, width/2, height/2, cameras[0]);
opencv = new OpenCV(this, width/2, height/2);
opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);
video.start();
////////////////////////////////////////////////////
port = new Serial(this, "COM7", 57600); //Baud rate is set to 57600 to match the Arduino baud rate.
//Send the initial pan/tilt angles to the Arduino to set the device up to look straight forward.
port.write(tiltChannel); //Send the Tilt Servo ID
port.write(servoTiltPosition); //Send the Tilt Position (currently 90 degrees)
port.write(panChannel); //Send the Pan Servo ID
port.write(servoPanPosition); //Send the Pan Position (currently 90 degrees)
}
void draw() {
scale(2);
opencv.loadImage(video);
image(video, 0, 0 );
noFill();
stroke(0, 255, 0);
strokeWeight(3);
Rectangle[] faces = opencv.detect();
println(faces.length);
for (int i = 0; i < faces.length; i++) {
println(faces[i].x + "," + faces[i].y);
rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
}
///////////////////////////////////////////
//Find out if any faces were detected.
if(faces.length > 0){
//If a face was found, find the midpoint of the first face in the frame.
//NOTE: The .x and .y of the face rectangle corresponds to the upper left corner of the rectangle,
// so we manipulate these values to find the midpoint of the rectangle.
midFaceY = faces[0].y + (faces[0].height/2);
midFaceX = faces[0].x + (faces[0].width/2);
//Find out if the Y component of the face is below the middle of the screen.
if (midFaceY > (midScreenY + midScreenWindow)) {
if (servoTiltPosition >= 5)servoTiltPosition -= stepSize; //If it is below the middle of the screen, update the tilt position variable to lower the tilt servo.
}
//Find out if the Y component of the face is above the middle of the screen.
else if (midFaceY < (midScreenY - midScreenWindow)) {
if (servoTiltPosition <= 175)servoTiltPosition +=stepSize; //Update the tilt position variable to raise the tilt servo.
}
//Find out if the X component of the face is to the left of the middle of the screen.
if(midFaceX < (midScreenX - midScreenWindow)){
if(servoPanPosition >= 5)servoPanPosition -= stepSize; //Update the pan position variable to move the servo to the left.
}
//Find out if the X component of the face is to the right of the middle of the screen.
else if(midFaceX > (midScreenX + midScreenWindow)){
if(servoPanPosition <= 175)servoPanPosition +=stepSize; //Update the pan position variable to move the servo to the right.
}
}
//Update the servo positions by sending the serial command to the Arduino.
port.write(tiltChannel); //Send the tilt servo ID
port.write(servoTiltPosition); //Send the updated tilt position.
port.write(panChannel); //Send the Pan servo ID
port.write(servoPanPosition); //Send the updated pan position.
if(faces.length == 0){
servoTiltPosition = 90;
servoPanPosition = 90;
}
delay(1);
}
void captureEvent(Capture c) {
c.read();
}