In this post I am going to show how to build a 2WD voice-controlled robot. Although in this post I am controlling only DC motors to move the robot, the same approach can be used to voice-control stepper motors and servo motors, so if you are planning on building any voice-controlled robot that perform mechanical movements, this post can serve as reference for your project.
To build this robot, you will basically need one 2-wheel chassis, one Arduino board, one DC motor driver shield, one BitVoicer Server license, one WiFi module and one pre-amplified microphone. Regarding the components, there is a huge variety of them in the market and you can achieve the same results using different combinations. In Step 1, I give a few details about some components used in the robot, so if you need to change some of them, just look for components with similar features.
The following procedures will be executed to transform voice commands into robot movements:
Audio waves will be captured and amplified by the Sparkfun Electret Breakout board;
In this step I give some important information about the robot’s components and how to prepare them for mounting.
In the first place, the heart of the robot: an Arduino UNO R3. Although I am using an Arduino UNO, you can use other Arduino boards to mount your robot. I decided to use an Arduino UNO because it is, by far, the most popular Arduino board and more people would be able to rebuild this robot. If you have an Arduino DUE, you can also add voice responses to your robot as I did in this post.
To move the DC motors, I used this Pololu shield. It is a dual motor driver capable on controlling 5 to 28V DC motors and deliver up to 3A continuous current per motor. Although I think this is a very good shield, what impressed me the most in Pololu’s products is their clear and detailed documentation. Besides, Pololu provides an Arduino library that makes really simple to control the motors. You just have to pass the speed and direction (negative or positive values) to the setSpeeds function. If you opt to use another DC motor driver, pay attention to the pins used by the driver because they cannot conflict with any other pin used by the robot. The Pololu shield uses the following Arduino pins: digital 4, 7, 8, 9, 10 and 12; analogic 0 and 1; ground; and 5V. The most important aspect here is that only pins 9 and 10 are used as PWM pins and the timer used to generate pulses on these pins is not the same timer (timer 2 on the UNO) used by the BVSMic class from BitSophia.
To establish communication with the server and access BitVoicer Server services, I use the Microchip WiFi RN171VX module. Two main features stand out in this module: all communication can be done through the serial port in a simple manner; and the module has the same form factor as most of the popular XBee modules. In other words, if you already have an XBee shield, you most likely will not need to buy the Sparkfun shield I use in this robot.
Using the XBee shield with the Microchip WiFi module made mounting the robot pretty easy. However, I have identified a problem in this combination. It is known that working with radio is extremely complex and considered by some people as “voodoo magic”. Well, I noticed that while the WiFi adapter is on and transmitting data, a strong interference occurs in the audio signal measured by the Arduino ADC. I was not able to precisely identify the source of this interference, but I have two possible sources: the module peak current draws (up to 240mA) are not being properly decoupled and compromise the reference voltage provided to the Arduino ADC; or the signal emitted by the antenna is being picked up by some exposed pins right below the module. Usually, WiFi module manufacturers instruct designers to avoid placing anything (even ground planes) near the antenna precisely to prevent this kind of issue.
To correct the problem described above, I soldered a 3.3V voltage regulator and one 10μF electrolytic capacitor to the Sparkfun shield so they could provide the reference voltage to the Arduino ADC. The same power source is used by the Sparkfun electret microphone. It solved the peak volume problems I was seeing in the BitVoicer Server Manager. Even during silence periods, the audio level was getting as high as 35 (0-100) in the Server Monitor. If you see the same thing happening in your project, investigate what is going on with the audio signal measured by the Arduino ADC.
I decided I was also going to use the prototyping space available in the Sparkfun shield to solder a few LEDs that tell the status of some BitVoicer Server functionalities. In the picture below, from the left to the right, the following information is exposed by the LEDs:
Indicates whether BitVoicer Server is running and/or if the connection remains active;
Indicates whether a speech recognition engine has been assigned to the Arduino;
Indicates whether we are in the activation-word-activated-period. This LED lights up only when the activation word is identified.
Regarding the preamplified electret microphone, there are many options available in the market: Sparkfun, Adafruit, RoboCore, Hackerstore and probably many others. To use the code I am posting on this tutorial, make sure the microphone you are buying is analogic, the required voltage is available in your Arduino board, and the amplification is high enough (usually 100x for electret microphones) for the Arduino ADC.
The 2WD chassis used in the robot is one of the cheapest and most popular on ebay. The complete kit contains one acrylic platform, two plastic/rubber wheels, one 360º wheel, two DC motors with gear (1:48 ratio), one 4xAA battery holder, two speed encoder disks and a set of screws.
Let’s get to work! In this step you have to mount all components on the chassis as shown in the pictures below.
If you do not want to solder the LEDs and the voltage regulator on the XBee shield, you can mount them on a small breadboard as shown in the picture below. In this case, choose a reduced size breadboard so you can fix it somewhere on the chassis without further problems.
In the pictures above, you can note that I made an iron wire support to hold the microphone. You can also see that I glued a strip of felted fabric around the 360º wheel. I took these two measures to try to reduce the noise generated by the robot and captured by the microphone when the robot is moving. I even cut a small piece of sponge to isolate the microphone wires from the support. Of course it was not 100% effective, but it reduced the noise a little bit and improved the accuracy of the speech recognition.
As I said in Step 1, the Microchip RN171VX WiFi module can be totally operated through the serial port. This makes extremely easy to set up the module because all commands are simple strings sent through the Arduino serial port. You just have to send a string containing “$$$” to enter command mode, send the commands and then send a string containing “exit” to return to data mode.
The code below is the code I used to set up the WiFi module in my home network. You will have to change the parts of the code marked as “XXXXXX” with information from your WiFi network. Three commands will have to be changed:
set wlan ssid XXXXXX: replace the XXXXXX with your network name (SSID);
set wlan phrase XXXXXX: replace the XXXXXX with your network password;
set ip address XXXXXX: replace the XXXXXX with the IP address (static) you wish to set to your WiFi module.
void setup()
{
Serial.begin(9600);
pinMode(13, OUTPUT);
delay(5000);
Serial.print("$$");
delay(1000);
Serial.println("set wlan auth 4");
delay(1000);
Serial.println("set wlan phrase XXXXXX");
delay(1000);
Serial.println("set wlan ssid XXXXXX");
delay(1000);
Serial.println("set wlan channel 0");
delay(1000);
Serial.println("set wlan join 1");
delay(1000);
Serial.println("set wlan tx 0");
delay(1000);
Serial.println("set ip dhcp 0");
delay(1000);
Serial.println("set ip address XXXXXX");
delay(1000);
Serial.println("set comm remote 0");
delay(1000);
Serial.println("set comm close 0");
delay(1000);
Serial.println("set comm open 0");
delay(1000);
Serial.println("set comm size 500");
delay(1000);
Serial.println("set comm time 50");
delay(1000);
Serial.println("set uart baud 115200");
delay(1000);
Serial.println("set uart flow 0");
delay(1000);
Serial.println("save");
delay(1000);
Serial.println("exit");
delay(1000);
digitalWrite(13, LOW);
}
void loop() { }
In my WiFi network, the authentication method is WPA2-PSK. If your network uses a different authentication method, you will also have to change the set wlan auth command. Check out the WiFi module documentation (section 4.3 Set Commands) to find out which value is the correct one for your network.
There is an important detail about the Sparkfun shield and its small switch (picture below). To upload code to the Arduino using its USB interface, the switch must be set to the DLINE position. For the Arduino to send/receive data through the WiFi module using its serial port, the switch must be set to the UART position. This is necessary because the WiFi module and the USB chip on the Arduino UNO use the same serial port in the ATmega microcontroller. If the switch is set to UART and you try to upload code to the Arduino, an error message will be displayed in the Arduino IDE.
To upload the code above to the Arduino and allow the Arduino to set up the WiFi module, follow the steps below:
Set the switch to the DLINE position;
Open the Arduino IDE, paste the code above into it and upload the code;
As soon as the upload finishes, you will have 5 seconds (delay at the beginning of the code) to change the switch to the UART position before commands are sent to the WiFi module. If the first command is lost, none of the others will work. In this case, simply reset the Arduino so the sketch can run again from the beginning.
During the module configuration, which takes about 25 seconds, the module LEDs will blink differently from its standard pattern. At this moment you will know the WiFi module is being configured.
After the module is configured, try to ping (Command Prompt --> “ping [IP Address]” --> press Enter) the module using the IP address specified in the set ip address command. If you do not get a response from the module, something went wrong in the previous steps.
Although the robot has only two DC motors, it is capable of performing a series of complex movements. To keep this post as simple as possible, I chose to define only about three dozens of basic uniform movements and a few complex movements formed by the combination of the basic movements.
As you can see in the pictures in Step 2, I do not use rotation sensors in the wheels, ultrasonic sensors or any other type of sensor to measure the distance from objects or the traveled distance. This prevents the robot from performing high precision movements. However, controlling only the direction and the speed of the motors you can achieve a level of precision good enough to move the robot around.
The first thing you need to know to be able to calculate the necessary execution time for each movement is the robot average speed. To do that, place a tape measure parallel to the robot and activate both motors simultaneously for one or two seconds, measure the traveled distance and deduce the speed. In my configuration, I got 13.7 centimeters per second employing 62.5% of the maximum motor speed (250/400, see Pololu Arduino library). In other words, to move the robot 1 meter (100 cm) ahead, the motors had to be activated simultaneously for 7.299270… seconds. I have chosen to keep the time counting in the milliseconds resolution, but if you want to achieve higher movement precision, consider raising the resolution to microseconds. Long story short, to move the robot 1 meter, I have to activate both motors simultaneously for 7299 milliseconds. From this number, everything becomes rule of three for other distances. To perform arc or circular movements, one wheel has to move faster than the other. To turn the robot to the sides, only one wheel has to be activated or both in opposite directions (to turn on its own axis). Here you will have to use some trigonometry to figure out the distance traveled by each wheel and for how long each motor must be activated. A good starting point for these concepts can be found in the following links (I do not intend to go further on this here): A Path Based on an Arc of a Circle and geometry - Move two wheeled robot from one point to another - Mathematics Stack Exchange.
As you can see at the end of the video below, I also make the robot “draw” some basic geometric forms (square, triangle and circle) on the floor. These movements are achieved by the combination of basic movements (e.g. go forward, turn, go forward, turn, etc.). The combination of these movements is made in the BitVoicer Server Voice Schema and you will not see them in the Arduino sketch presented in the next step.
In this step you will have to upload the code below to the Arduino. You can also download the Arduino sketch from the link below the code. Remember that to send code to the Arduino you must set the switch on the Sparkfun shield to the DLINE position as described in Step 3. Before you upload the code, you must properly install the BitVoicer Server and the Pololu motor driver libraries into the Arduino IDE (Importing a .zip Library).
#include <BVSP.h>
#include <BVSMic.h>
#include <DualMC33926MotorShield.h>
// Defines the Arduino pins that will be used to control
// LEDs and capture audio
#define BVS_RUNNING 2
#define BVS_SRE 5
#define BVS_DATA_FWD 3
#define BVS_ACT_PERIOD 6
#define BVSM_AUDIO_INPUT 3
// Defines the constants that will be passed as parameters to
// the BVSP.begin function
const unsigned long STATUS_REQUEST_INTERVAL = 2000;
const unsigned long STATUS_REQUEST_TIMEOUT = 1000;
// Defines the size of the mic buffer
const int MIC_BUFFER_SIZE = 64;
// Initializes a new global instance of the BVSP class
BVSP bvsp = BVSP();
// Initializes a new global instance of the BVSMic class
BVSMic bvsm = BVSMic();
// Initializes a new global instance of the
// DualMC33926MotorShield class
DualMC33926MotorShield ms = DualMC33926MotorShield();
// Creates a buffer that will be used to read recorded samples
// from the BVSMic class
byte micBuffer[MIC_BUFFER_SIZE];
// Creates a global variable that indicates whether the
// Arduino is connected to BitVoicer Server
boolean connected = false;
// Defines some constants for the motor settings
const int SPEED_STOP = 0;
const int SPEED_SLOW = 100;
const int SPEED_NORMAL = 250;
const int SPEED_FAST = 400;
const int DIRECTION_FRONT = -1;
const int DIRECTION_BACK = 1;
// Declares a global variables to hold the current motor speed.
// The default is SPEED_NORMAL, but there are voice
// commands that change this setting.
int motorSpeed = SPEED_NORMAL;
// Stores the command duration in milliseconds
unsigned long cmdDuration = 0;
// Stores the time the command started running
unsigned long cmdStartTime = 0;
// Stores whether a command is running or not
bool cmdRunning = false;
// Stores the last MOVE_FORWARD command. This variable
// is used only for the COME_BACK command.
byte lastFwdCmd = 0;
// Defines some constants for command names/values
// Just to make the code more readable
const byte CMD_STOP = 0;
const byte CMD_MOVE_FORWARD = 1;
const byte CMD_MOVE_FORWARD_1_CM = 2;
const byte CMD_MOVE_FORWARD_2_CM = 3;
const byte CMD_MOVE_FORWARD_5_CM = 4;
const byte CMD_MOVE_FORWARD_10_CM = 5;
const byte CMD_MOVE_FORWARD_25_CM = 6;
const byte CMD_MOVE_FORWARD_50_CM = 7;
const byte CMD_MOVE_FORWARD_1_M = 8;
const byte CMD_MOVE_BACKWARD = 9;
const byte CMD_MOVE_BACKWARD_1_CM = 10;
const byte CMD_MOVE_BACKWARD_2_CM = 11;
const byte CMD_MOVE_BACKWARD_5_CM = 12;
const byte CMD_MOVE_BACKWARD_10_CM = 13;
const byte CMD_MOVE_BACKWARD_25_CM = 14;
const byte CMD_MOVE_BACKWARD_50_CM = 15;
const byte CMD_MOVE_BACKWARD_1_M = 16;
const byte CMD_TURN_AROUND = 17;
const byte CMD_TURN_AROUND_RIGHT = 18;
const byte CMD_TURN_AROUND_LEFT = 19;
const byte CMD_DO_360 = 20;
const byte CMD_TURN_RIGHT = 21;
const byte CMD_TURN_RIGHT_10 = 22;
const byte CMD_TURN_RIGHT_25 = 23;
const byte CMD_TURN_RIGHT_45 = 24;
const byte CMD_TURN_LEFT = 25;
const byte CMD_TURN_LEFT_10 = 26;
const byte CMD_TURN_LEFT_25 = 27;
const byte CMD_TURN_LEFT_45 = 28;
const byte CMD_DO_CIRCLE = 29;
const byte CMD_COME_BACK = 30;
const byte CMD_MOVE_FORWARD_2_M = 31;
const byte CMD_MOVE_FORWARD_3_M = 32;
const byte CMD_MOVE_BACKWARD_2_M = 33;
const byte CMD_MOVE_BACKWARD_3_M = 34;
const byte CMD_SET_SPEED_SLOW = 35;
const byte CMD_SET_SPEED_NORMAL = 36;
const byte CMD_SET_SPEED_FAST = 37;
const byte CMD_TURN_LEFT_45_BACKWARD = 38;
const byte CMD_TURN_RIGHT_45_BACKWARD = 39;
void setup()
{
// Starts serial communication at 115200 bps
Serial.begin(115200);
// Sets the Arduino pin modes
pinMode(BVS_RUNNING, OUTPUT);
pinMode(BVS_SRE, OUTPUT);
pinMode(BVS_DATA_FWD, OUTPUT);
pinMode(BVS_ACT_PERIOD, OUTPUT);
AllLEDsOff();
// Sets the Arduino serial port that will be used for
// communication, how long it will take before a status request
// times out and how often status requests should be sent to
// BitVoicer Server
bvsp.begin(Serial, STATUS_REQUEST_TIMEOUT,
STATUS_REQUEST_INTERVAL);
// Sets the function that will handle the frameReceived
// event
bvsp.frameReceived = BVSP_frameReceived;
// Prepares the BVSMic class timer
bvsm.begin();
// Prepares the motor shield class (pins and timer1)
ms.init();
}
void loop()
{
// If it is not connected to the server, opens a TCP/IP
// connection, sets connected to true and resets the BVSP
// class
if (!connected)
{
Connect(Serial);
connected = true;
bvsp.reset();
}
// Checks if the status request interval has elapsed and if it
// has, sends a status request to BitVoicer Server
bvsp.keepAlive();
// Checks if there is data available at the serial port buffer
// and processes its content according to the specifications
// of the BitVoicer Server Protocol
bvsp.receive();
// Gets the respective status from the BVSP class and sets
// the LEDs on or off
digitalWrite(BVS_RUNNING, bvsp.isBVSRunning());
digitalWrite(BVS_DATA_FWD, bvsp.isDataFwdRunning());
// Checks if there is a SRE assigned to the Arduino
if (bvsp.isSREAvailable())
{
// Turns on the SRE available LED
digitalWrite(BVS_SRE, HIGH);
// If the BVSMic class is not recording, sets up the audio
// input and starts recording
if (!bvsm.isRecording)
{
bvsm.setAudioInput(BVSM_AUDIO_INPUT, EXTERNAL);
bvsm.startRecording();
}
// Checks if the BVSMic class has available samples
if (bvsm.available)
{
// Makes sure the inbound mode is STREAM_MODE before
// transmitting the stream
if (bvsp.inboundMode == FRAMED_MODE)
bvsp.setInboundMode(STREAM_MODE);
// Reads the audio samples from the BVSMic class
int bytesRead = bvsm.read(micBuffer, MIC_BUFFER_SIZE);
// Sends the audio stream to BitVoicer Server
bvsp.sendStream(micBuffer, bytesRead);
}
}
else
{
// There is no SRE available
// Turns off the SRE and ACT_PERIOD LEDs
digitalWrite(BVS_SRE, LOW);
digitalWrite(BVS_ACT_PERIOD, LOW);
// If the BVSMic class is recording, stops it
if (bvsm.isRecording)
bvsm.stopRecording();
}
// If the status has timed out, the connection is considered
// lost
if (bvsp.hasStatusTimedOut())
{
// If the BVSMic is recording, stops it
if (bvsm.isRecording)
bvsm.stopRecording();
// Closes the TCP/IP connection
Disconnect(Serial);
AllLEDsOff();
connected = false;
}
// If a command is running, checks if its duration has
// expired. If it has, stop the motors.
if (cmdRunning)
if (millis() - cmdStartTime >= cmdDuration)
RunCommand(CMD_STOP);
}
// Handles the frameReceived event
void BVSP_frameReceived(byte dataType, int payloadSize)
{
// Performs the appropriate actions based on the frame
// data type. If the data type is byte, it is a command.
// If the data type is int, changes the activated
// period LED.
switch (dataType)
{
case DATA_TYPE_BYTE:
RunCommand(bvsp.getReceivedByte());
break;
case DATA_TYPE_INT16:
digitalWrite(BVS_ACT_PERIOD, bvsp.getReceivedInt16());
break;
}
}
// Continue ...
Some parts of this sketch are similar to parts I used in one of my previous posts and deals with the communication with BitVoicer Server (BVSP and BVSMic classes). In this post, I will stick to the explanation of the new parts of the sketch. If you want to get more information about how to use the BVSP and the BVSMic classes, I suggest your refer to the post I mentioned above.
Constants Declaration: at the beginning of the sketch, I declare a series of constants used throughout the code. The group of constants with motor settings defines the default motor speeds and two direction constants. The Pololu motor driver library accepts values from -400 to +400 for the motor speed where zero means off. Negative values indicate reverse rotation or, if you have inverted the motor wires like me, forward rotation. The group of constants with command values refers to the commands that will be sent from BitVoicer Server. In this sketch I have defined only 40 basic commands of byte type, but more complex movements can be performed combining these commands.
Execution Control Variables: five variables are defined at the beginning of the code to control the execution of commands (motorSpeed, cmdDuration, cmdStartTime, cmdRunning e lastFwdCmd). The motorSpeed variable holds the current motor speed. This variable is updated with one of the default values defined by the speed constants if the Arduino receives a command from BitVoicer Server to update the motor speeds. The cmdDuration variable holds the total duration of the current command. This variable is checked against the cmdStartTime variable in all iterations of the loop function if cmdRunning is true. If the command execution time has expired, the RunCommand function is called to stop the motors. The lastFwdCmd variable holds the last “go/move forward” command. This variable is used to know the last traveled distance so the “come back” command can be executed. Note that to use this command you must first tell the robot to turn around.
WiFi Connection: at the end of the sketch I define two functions to connect and disconnect from BitVoicer Server (Connect and Disconnect). These functions put the Microchip WiFi module into command mode, open or close a TCP/IP connection and return the module into data mode. Inside the loop function, if the connected variable is not true, I call the Connect function. If the BVSP class reports the server status has expired, in other words, no answer has been received for the last status request, I assume the connection has been lost and I call the Disconnect function. This will force a new connection attempt in the next loop iteration.
RunCommand function: this function is called every time a command is received from BitVoicer Server. It takes a byte value that corresponds to one of the basic commands defined by the constants at the beginning of the sketch. Each basic command is identified inside the switch so the appropriate motor speeds can be set as well as the command duration. At the end of the function, the cmdRunning variable is set to true and the time returned by the millis function is stored in the cmdStartTime variable. This allows the Arduino to control the command execution as described above. The times, in milliseconds, for each command were obtained as described in the previous step.
In this step you have to build the BitVoicer Server Voice Schema with the sentences to be recognized and the commands that will be sent to the Arduino. At the end of this step, there are two links to files containing all BitVoicer Server Solution Objects used in this post. If you do not want to create all solution objects one-by-one, you can import them (Importing Solution Objects) using these files.
Before you start building the Voice Schema, you have to create a device that will represent the Arduino in BitVoicer Server. Create a mixed device and name it ArduinoUnoWiFi. In the Communication tab, select TCP/IP and enter the IP address assigned to the WiFi module in Step 3. In the Cues tab, enable the Start of Activated Period and End of Activated Period cues. Select Int16 SendData commands for both cues and select the ArduinoUnoWiFi device as the target of the commands. In the Data field, enter 1 for the Start of Activated Period cue and 0 for the End of Activated Period cue. These cues will make the Arduino turn on one LED every time the activation word is recognized. When the activated period (defined in the Voice Schema) expires, this LED will turn off.
Now let’s build the Voice Schema. Although the robot has only 40 basic commands, you may want to create many word combinations that trigger the same command. As an example, you may want the sentences “move forward one meter” and “go forward one meter” to trigger the same command. Besides, you may want to create complex commands as “do a square” that will trigger a series of temporized commands in sequence. Because of that, your Voice Schema can grow a lot and have many more than 40 basic commands. BitVoicer Server supports unlimited sentences so you can define as many sentences as you need (mine ended up with more than 80 sentences). Here I would like to give two tips: create a default command in the BitVoicer Server Manager Options; and copy and paste duplicated commands from one sentence to the others.
Sentence commands will send byte data types to the Arduino. Use the constants defined at the beginning of the sketch to know which value must be sent for each command. Complex commands will send many values in sequence and you will have to control the interval (delay) between them so that no value is sent while the previous command is under execution. Use the delay field to set the interval between commands.
Now all you have to do is to put in the AA batteries and connect the 9V battery to the Arduino power jack for your robot to come to life! After you upload the code to the Arduino, do not forget to turn the switch mounted on the Sparkfun shield to the UART position. Otherwise, the WiFi module will not receive any data sent from the Arduino serial port.
After you turn the robot on, the status LEDs will take a little while before they light up. You will see the WiFi module starting communication and when the TCP/IP connection is established, one of the module LEDs will turn solid on. After a few instants, three out of the four status LEDs will light up as well. This means one speech recognition engine has been assigned to the Arduino. From now on, the robot is ready to take commands.
After some testing with the robot, I was pretty satisfied with the speech recognition, although it did not recognize 100% of the commands all the times. In this aspect BitVoicer Server really surprised me. However, I was not that much satisfied with the precision of the robot’s movements. To fix this problem, I would have to add rotation sensors to the wheels. The chassis kit I used in the robot already comes with decoder disks that can be attached to the gear. Applying sensors to these disks would allow me to move the robot based on the real traveled distance so that its movement would be more precise. I could also add ultrasonic sensors to avoid bumping around my furniture. I will probably do that someday, but for now, I leave that up to you.
