Voice recognition to Arduino to processing communication issue

Hi everyone. This is my first post here. I hope I write clearly :expressionless:

I want to use arduino, connected to a voice recognition module, to play and pause an .mp4 file on my computer through processing.

Tools I am using:

  1. Arduino Uno
  2. Elechouse VR3 voice recognition module
  3. Processing 3.3.7

Steps I took:

I first trained the VR module. I uploaded the sample train program onto arduino and used the serial monitor to teach it two words on records 0 and 1. Then loaded those records onto the VR module.

I used the sample LED code given with the VR manual to check if it was trained properly. I uploaded the code onto the arduino and LED 13 on the arduino would switch on and off with my words.

I changed some parts of the sample code like this:

/**
  ******************************************************************************
  * @file    vr_sample_control_led.ino
  * @author  JiapengLi
  * @brief   This file provides a demostration on 
              how to control led by using VoiceRecognitionModule
  ******************************************************************************
  * @note:
        voice control led
  ******************************************************************************
  * @section  HISTORY
    
    2013/06/13    Initial version.
  */
  
#include <SoftwareSerial.h>
#include "VoiceRecognitionV3.h"

/**        
  Connection
  Arduino    VoiceRecognitionModule
   2   ------->     TX
   3   ------->     RX
*/
VR myVR(2,3);    // 2:RX 3:TX, you can choose your favourite pins.

uint8_t records[7]; // save record
uint8_t buf[64];

#define karachi    (0)
#define end   (1) 

/**
  @brief   Print signature, if the character is invisible, 
           print hexible value instead.
  @param   buf     --> command length
           len     --> number of parameters
*/
void printSignature(uint8_t *buf, int len)
{
  int i;
  for(i=0; i<len; i++){
    if(buf[i]>0x19 && buf[i]<0x7F){
      Serial.write(buf[i]);
    }
    else{
      Serial.print("[");
      Serial.print(buf[i], HEX);
      Serial.print("]");
    }
  }
}

/**
  @brief   Print signature, if the character is invisible, 
           print hexible value instead.
  @param   buf  -->  VR module return value when voice is recognized.
             buf[0]  -->  Group mode(FF: None Group, 0x8n: User, 0x0n:System
             buf[1]  -->  number of record which is recognized. 
             buf[2]  -->  Recognizer index(position) value of the recognized record.
             buf[3]  -->  Signature length
             buf[4]~buf[n] --> Signature
*/
void printVR(uint8_t *buf)
{
  Serial.println("VR Index\tGroup\tRecordNum\tSignature");

  Serial.print(buf[2], DEC);
  Serial.print("\t\t");

  if(buf[0] == 0xFF){
    Serial.print("NONE");
  }
  else if(buf[0]&0x80){
    Serial.print("UG ");
    Serial.print(buf[0]&(~0x80), DEC);
  }
  else{
    Serial.print("SG ");
    Serial.print(buf[0], DEC);
  }
  Serial.print("\t");

  Serial.print(buf[1], DEC);
  Serial.print("\t\t");
  if(buf[3]>0){
    printSignature(buf+4, buf[3]);
  }
  else{
    Serial.print("NONE");
  }
  Serial.println("\r\n");
}

void setup()
{
  /** initialize */
  myVR.begin(9600);
  
  Serial.begin(9600);
  Serial.println("Elechouse Voice Recognition V3 Module\r\nControl LED sample");
    
  if(myVR.clear() == 0){
    Serial.println("Recognizer cleared.");
  }else{
    Serial.println("Not find VoiceRecognitionModule.");
    Serial.println("Please check connection and restart Arduino.");
    while(1);
  }
  
  if(myVR.load((uint8_t)karachi) >= 0){
    Serial.println("karachi loaded");
  }
  
  if(myVR.load((uint8_t)end) >= 0){
    Serial.println("end loaded");
  }
}

void loop()
{
  int ret;
  ret = myVR.recognize(buf, 50);
  if(ret>0){
    switch(buf[1]){
      case karachi:
        /** turn on LED */
        Serial.println("karachi");
        break;
      case end:
        /** turn off LED*/
        Serial.write ("end");
        break;
      default:
        Serial.write("Record function undefined");
        break;
    }
    /** voice recognized */
    printVR(buf);
  }
}

then I wrote a code on processing to make the movie open and play and pause based on communication from the VR and arduino. this was my processing code:

import processing.serial.*;
import processing.video.*;
String PATH = "/Users/kehkashankhalid/Desktop/totoro.mp4";
Movie mov;

Serial myPort;  // Create object from Serial class
String val;     // Data received from the serial port

void setup()
{
  String portName = Serial.list()[1]; //change the 0 to a 1 or 2 etc. to match your port
  myPort = new Serial(this, "/dev/cu.usbmodem1421", 115200);
  size(384, 750);
  frameRate(30);
  mov = new Movie(this, PATH);

}

void draw()
{
  if ( myPort.available() > 0) 
  {  // If data is available,
  val = myPort.readString();         // read it and store it in val
  } 
println(val); //print it out in the console
image(mov, 0, 0, width, height);
if (val == "karachi"){
    mov.play();
}

else if (val == "end"){
    mov.pause();
}

}

void movieEvent(Movie m) {
  m.read();
}

But in the end the arduino serial monitor shows all thelines but does not react when I say my chosen words even though it reacted when i used the LED code. Meanwhile Processing opens a window of the appropriate size but doesn’t play a movie at all, let alone pause it. Processing also does not react to the voice commands.

Could you help me correct my code?

Hi everyone. This is my first post here. I hope I write clearly :expressionless:

I want to use arduino, connected to a voice recognition module, to play and pause an .mp4 file on my computer through processing.

Tools I am using:

  1. Arduino Uno
  2. Elechouse VR3 voice recognition module
  3. Processing 3.3.7

Steps I took:

I first trained the VR module. I uploaded the sample train program onto arduino and used the serial monitor to teach it two words on records 0 and 1. Then loaded those records onto the VR module.

I used the sample LED code given with the VR manual to check if it was trained properly. I uploaded the code onto the arduino and LED 13 on the arduino would switch on and off with my words.

I changed some parts of the sample code like this:

/**
  ******************************************************************************
  * @file    vr_sample_control_led.ino
  * @author  JiapengLi
  * @brief   This file provides a demostration on 
              how to control led by using VoiceRecognitionModule
  ******************************************************************************
  * @note:
        voice control led
  ******************************************************************************
  * @section  HISTORY
    
    2013/06/13    Initial version.
  */
  
#include <SoftwareSerial.h>
#include "VoiceRecognitionV3.h"

/**        
  Connection
  Arduino    VoiceRecognitionModule
   2   ------->     TX
   3   ------->     RX
*/
VR myVR(2,3);    // 2:RX 3:TX, you can choose your favourite pins.

uint8_t records[7]; // save record
uint8_t buf[64];

#define karachi    (0)
#define end   (1) 

/**
  @brief   Print signature, if the character is invisible, 
           print hexible value instead.
  @param   buf     --> command length
           len     --> number of parameters
*/
void printSignature(uint8_t *buf, int len)
{
  int i;
  for(i=0; i<len; i++){
    if(buf[i]>0x19 && buf[i]<0x7F){
      Serial.write(buf[i]);
    }
    else{
      Serial.print("[");
      Serial.print(buf[i], HEX);
      Serial.print("]");
    }
  }
}

/**
  @brief   Print signature, if the character is invisible, 
           print hexible value instead.
  @param   buf  -->  VR module return value when voice is recognized.
             buf[0]  -->  Group mode(FF: None Group, 0x8n: User, 0x0n:System
             buf[1]  -->  number of record which is recognized. 
             buf[2]  -->  Recognizer index(position) value of the recognized record.
             buf[3]  -->  Signature length
             buf[4]~buf[n] --> Signature
*/
void printVR(uint8_t *buf)
{
  Serial.println("VR Index\tGroup\tRecordNum\tSignature");

  Serial.print(buf[2], DEC);
  Serial.print("\t\t");

  if(buf[0] == 0xFF){
    Serial.print("NONE");
  }
  else if(buf[0]&0x80){
    Serial.print("UG ");
    Serial.print(buf[0]&(~0x80), DEC);
  }
  else{
    Serial.print("SG ");
    Serial.print(buf[0], DEC);
  }
  Serial.print("\t");

  Serial.print(buf[1], DEC);
  Serial.print("\t\t");
  if(buf[3]>0){
    printSignature(buf+4, buf[3]);
  }
  else{
    Serial.print("NONE");
  }
  Serial.println("\r\n");
}

void setup()
{
  /** initialize */
  myVR.begin(9600);
  
  Serial.begin(9600);
  Serial.println("Elechouse Voice Recognition V3 Module\r\nControl LED sample");
    
  if(myVR.clear() == 0){
    Serial.println("Recognizer cleared.");
  }else{
    Serial.println("Not find VoiceRecognitionModule.");
    Serial.println("Please check connection and restart Arduino.");
    while(1);
  }
  
  if(myVR.load((uint8_t)karachi) >= 0){
    Serial.println("karachi loaded");
  }
  
  if(myVR.load((uint8_t)end) >= 0){
    Serial.println("end loaded");
  }
}

void loop()
{
  int ret;
  ret = myVR.recognize(buf, 50);
  if(ret>0){
    switch(buf[1]){
      case karachi:
        /** turn on LED */
        Serial.println("karachi");
        break;
      case end:
        /** turn off LED*/
        Serial.write ("end");
        break;
      default:
        Serial.write("Record function undefined");
        break;
    }
    /** voice recognized */
    printVR(buf);
  }
}

then I wrote a code on processing to make the movie open and play and pause based on communication from the VR and arduino. this was my processing code:

import processing.serial.*;
import processing.video.*;
String PATH = "/Users/kehkashankhalid/Desktop/totoro.mp4";
Movie mov;

Serial myPort;  // Create object from Serial class
String val;     // Data received from the serial port

void setup()
{
  String portName = Serial.list()[1]; //change the 0 to a 1 or 2 etc. to match your port
  myPort = new Serial(this, "/dev/cu.usbmodem1421", 115200);
  size(384, 750);
  frameRate(30);
  mov = new Movie(this, PATH);

}

void draw()
{
  if ( myPort.available() > 0) 
  {  // If data is available,
  val = myPort.readString();         // read it and store it in val
  } 
println(val); //print it out in the console
image(mov, 0, 0, width, height);
if (val == "karachi"){
    mov.play();
}

else if (val == "end"){
    mov.pause();
}

}

void movieEvent(Movie m) {
  m.read();
}

But in the end the arduino serial monitor shows all thelines but does not react when I say my chosen words even though it reacted when i used the LED code. Meanwhile Processing opens a window of the appropriate size but doesn’t play a movie at all, let alone pause it. Processing also does not react to the voice commands.

Could you help me correct my code?

I changed some parts of the sample code like this:

Which parts did you change? I don't want to guess.

What did the program output before you made those changes?

I don't know Processing. Can you add some code to it so that you can see each message that it receives?

I presume the Arduino function printVR() is intended to send the command to Processing to operate the movies.

My wild guess is that you are confusing Processing by sending various debugging messages to it such as

Serial.println("karachi");

...R

before the changes the LED would light on and off.

I removed

int led = 13 because I no longer wanted the LED to switch on and off

I removed

pinMode(led, OUTPUT)

I replaced

digitalWrite(led, LOW or HIGH) with Serial.write("word")

and now the serial monitor shows that the module and commands are loaded and ready but nothing happens when I say the voice commands.

processing only opens a window of the right size and stays grey.

okay so instead of Serial.write I should try printVR()?

Also if I should remove the debugging code, which parts should I remove?

DON'T DOUBLE POST.

I have already responded in your other Thread.

I am suggesting to the Moderator to merge them.

...R

I just didn't know which category it fit in.

I replied to your initial response. Could you please check it and let me know. thanks

k-khalid: okay so instead of Serial.write I should try printVR()?

Also if I should remove the debugging code, which parts should I remove?

I had not noticed the function that uses Serial.write() - what is its purpose?

The parts you should remove are all the parts that are confusing Processing - that is, if there are parts that are confusing Processing.

Have you added debugging code to Processing like I suggested?

In my PC programs that communicate with an Arduino I usually have some means to distinguish between data that the Arduino sends and debug messages that the Arduino sends. For example my Arduino program might begin all its debug messages with the word "DEBUG "

...R

the serial write will send the bytes from voice recognition to processing I am assuming.

I don't know what you mean by debugging code :(

k-khalid: the serial write will send the bytes from voice recognition to processing I am assuming.

Didn't you write the program?

There are two functions in your program one is called printVR() and the other is called printSignature() - what is each of them for?

I don't know what you mean by debugging code :(

I just mean things that you put in (usually Serial.print() statements) to help you identify what is causing a problem.

...R

I actually did not write the code from scratch. The sample was given in the manual of the voice recognition module.

So I figured almost everything out!

My VR is communicating with my arduino. When I say the voice commands for 'play' or 'pause' they print in the arduino serial monitor.

My arduino is also communicating with processing because the play and pause appears in the processing console as well when i say it.

The only problem I am having is that processing plays my movie, and registers receiving the voice commands but it does not play or pause the movie when the appropriate voice command is used :/

Here's the code for processing:

import processing.serial.*;
import processing.video.*;
String PATH = "/Users/kehkashankhalid/Desktop/totoro.mp4";
Movie mov;

Serial myPort;  // Create object from Serial class
String val;     // Data received from the serial port

void setup()
{
 mov = new Movie(this, PATH);
 String portName = Serial.list()[1]; //change the 0 to a 1 or 2 etc. to match your port
 myPort = new Serial(this, "/dev/cu.usbmodem1421", 115200);
  size(384, 750);
 frameRate(30);
 mov.loop();

}

void draw()
{
 image(mov, 0, 0, width, height);
   if ( myPort.available() > 0) 
 {  // If data is available,
 val = myPort.readString();         // read it and store it in val
 } 
 if (val == "play"){
   mov.play();
}
else if (val == "pause"){
   mov.pause();
}
println(val); //print it out in the console

}

void movieEvent(Movie m) {
 m.read();
}

k-khalid: The only problem I am having is that processing plays my movie, and registers receiving the voice commands but it does not play or pause the movie when the appropriate voice command is used :/

If Processing is receiving the commands correctly then your Arduino code is working and you need to get advice about Processing on a Processing Forum.

I don't know Processing.

...R

Unlike the Arduino, Serial I/O should NOT be done in the draw() method.

You should make sure that you are sending JUST ‚Äúplay‚ÄĚ and ‚Äúpause‚ÄĚ, NOT ‚Äúplay‚ÄĚ or ‚Äúplay‚ÄĚ or ‚Äúplay‚ÄĚ.

Actually, you SHOULD be sending ‚Äúplay!‚ÄĚ or ‚Äúpause!‚ÄĚ (or some other terminator) and using readStringUntil() rather than readString().