vocal emulation + Arduino

Forum 2005-2010 (read only) Software Interfacing
#1

Hi! I am Giovanni from Milan. I am workinng on a project for university graduation in NABA www.naba.it.

The project is a robot that with human speech speaks with quotations of artist and writers like Pessoa and McLuhan. I have finished the movement and inputs part of the project but I don't know how to get inputs from arduino (to a computer) and produce speech (with computer speakers), (computer is in the robot structure with batteries). Someone advise to me speech libraries for c. Is it possibile to include one of this library and write an array of quotations to read sometimes randomly?

Pleeeease help me my discussion is in February

#2

so the arduino is inside the robot - correct?

Is the computer connected to the arduino already? If not, maybe bluetooth is the simplest wireless connection. Then you could have a processing.org application on the PC that could call other code to do the speech fragments on demand (or just play mp3s).

I have seen speech synthesis code for the arduino that would leave out the PC entirely but you'd have to power the speakers somehow onboard the robot. http://blog.makezine.com/archive/2009/12/text_to_speech_theres_a_shield_for.html

#3

Arduino is inside the robot.

Arduino is connected with computer with usb for power (is it possible to use usb connection for input/output actions also?). Then How can I create this processing program? Do you know some "how to"/"hello world" tutorial?

Very nice shield but i can't find a webshop that sell it!!! I Can't build it I think...

#4

Arduino is connected with computer with usb for power (is it possible to use usb connection for input/output actions also?

yes sure, it just looks like a serial port to the pc. here is a pointer to processing.org http://processing.org/. It's a development environment for the pc and there are lots of examples of it talking to arduino. I believe there is speech software for processing - search on that site.

#5

I have to put in a processing program my arduino motion and sensor code and add text to speech with an array of quotaions and a directory of images to show on a screen randomly. Is it possible? Can you help me?

#6

The processing sketch will just contain the text to speech code and the image display code.

The Arduino sketch will need to send data to the serial port (that the processing application is connected to) that conveys enough information for the processing sketch to know what to say and what picture to display.

Unless there is some relationship between what the robot is doing, or where it is, and the quotes and pics, the simplest thing to have the Arduino do is to tell the processing app. that it needs to do/say something. This can be just a matter of writing a 1 to the serial port.

The processing application will see that there is data available, do or say some random thing, and then flush the serial buffer.

If there is a relationship, the Arduino could write a number or letter to the serial port. The processing app. will see that there is data, read that data, and do/say the right thing, based on the letter/number read.

As a University project, I suspect that you are expected to learn something by doing this. Having us write the code for you will not have that result.

#7

Yeah I totally agree with you, I will be happy to learn the first steps of processing that would then lead me to continue by myself in my experiments (and to produce something useful for people). Can you show me a “hello world” program to start triyng?
I took a look to reference and howtos of processing but i can’t produce by myself the communication code.

here it is the code that I build on arduino:

#include <Servo.h>

// words for clarity
#define LEFT 1
#define RIGHT 2
#define LEDBICOLOR 0
#define LEDTRICOLOR 1

// parameters
#define BUMPS_TO_STOP 5                 // number of obstacles to stop after
#define RANDOM_START_SECONDS 120        // mean time in seconds for random start
#define RANDOM_SOUND_SECONDS 30         // mean time in seconds for random sound
#define RANDOM_STEER_SECONDS 3          // mean time in seconds for random steering
#define BACKWARD_MSECONDS 800           // time to go back after hitting an obstacle in milliseconds
#define ROTATION_MSECONDS 500           // duration of the rotation in milliseconds
#define STEER_MSECONDS 1000             // duration of the random steering
#define SPEED 100                       // speed to go
#define STEER_PERCENT 3                 // percent of SPEED to set one of the servos to steer
#define SNDMEAN 150                     // number of samples to mean

// digital pins
#define ledPin1      2                  // LED 0 red
#define ledPin2      3                  // LED 0 green
#define ledPin3      4                  // LED 1 blue
#define ledPin4      5                  // LED 1 green
#define ledPin5      6                 // LED sensor
#define winkerPin1   7                  // winker LEFT
#define winkerPin2   8                  // winker RIGHT
#define servoPin1    9                  // servo LEFT
#define servoPin2   10                  // servo RIGHT
#define outSndPin   11                  // speaker
#define ledPin6     12                  // LED sensor

// analog pins
#define inSndPin     5                  // sound sensor

// variables
float actualSpeed = 0.0;                // percent of actual moving speed we are accelerating to
boolean moving = false;                 // stopped or moving
int speedLeft = 0;                      // speed we want on left servo
int speedRight = 0;                     // speed we want on right servo
Servo left;
Servo right;

int contact = 0;                        // did we hit? where?
int contactCount = 0;                   // number of hits since start
int contactStep = 0;                    // part of the hit manouver we are in
long contactTime = 0;                   // time of hitting
boolean steering = false;               // steering or going straight
boolean lightCheck = true;

long lightTime = 0;                     // time of last light check
long startTime = 0;                     // time of last random start
long steerTime = 0;                     // time of last random steering change
long soundTime = 0;                     // time of last random sound

int meanSnd[SNDMEAN];                   // past values of sound sensor
long meanSndCount = 0;                  // number of values stored


// setup
void setup() {
  pinMode(ledPin1, OUTPUT);
  pinMode(ledPin2, OUTPUT);
  pinMode(ledPin3, OUTPUT);
  pinMode(ledPin4, OUTPUT);
  pinMode(ledPin6, OUTPUT);
  pinMode(winkerPin1, INPUT);
  pinMode(winkerPin2, INPUT);
  left.attach(servoPin1);
  right.attach(servoPin2);
  pinMode(outSndPin, OUTPUT);
  
  randomSeed(analogRead(0));
  digitalWrite(ledPin6, LOW);
  Serial.begin(9600);
}


// winkers
int checkContact() {
  if(digitalRead(winkerPin1) == LOW)
    return LEFT;
  if(digitalRead(winkerPin2) == LOW)
    return RIGHT;

  return 0;
}


// servo
void servoContinue() {
  if(moving) {
    if((actualSpeed < 1)) {
      actualSpeed += 0.009;
    }
    if(steering && (steerTime < millis() - STEER_MSECONDS)) {
      speedLeft = speedRight = max(speedLeft, speedRight);
      steering = false;
      steerTime = millis();
    }
  }
  
  servoSpeed(LEFT, round(speedLeft * actualSpeed));
  servoSpeed(RIGHT, round(speedRight * actualSpeed));
}

void servoGo(int speed) {
  if(speed < 0) {
    setLed(LEDBICOLOR, 1);
  }
  else {
    setLed(LEDBICOLOR, 0);
  }
  
  speedLeft = speedRight = speed;
  servoReset();
}

void servoReset() {
  actualSpeed = 0;
  servoContinue();
}

void servoRotate(int side) {
  setLed(LEDBICOLOR, 2);
  
  switch(side) {
  case LEFT:
    speedLeft = 40;
    speedRight = -40;
    break;
  case RIGHT:
    speedLeft = -40;
    speedRight = 40;
    break;
  }
  servoReset();
}

void servoSpeed(int side, int speed) {
  speed = speed * 90 / 100;
  switch(side) {
  case LEFT:
    left.write(90 + speed);
    break;
  case RIGHT:
    right.write(90 - speed);
    break;
  }
}

void servoSteer(int side) {
  steering = true;
  steerTime = millis();
  
  switch(side) {
  case LEFT:
    speedLeft = round(speedLeft * STEER_PERCENT / 100.0);
    break;
  case RIGHT:
    speedRight = round(speedRight * STEER_PERCENT / 100.0);
    break;
  }
}


// LED
void setLed(int led, int state) {
  switch(led) {
  case LEDBICOLOR:
    switch(state) {
    case 0:
      digitalWrite(ledPin1, LOW);
      digitalWrite(ledPin2, LOW);
      break;
    case 1:
      digitalWrite(ledPin1, HIGH);
      digitalWrite(ledPin2, LOW);
      break;
    case 2:
      digitalWrite(ledPin1, LOW);
      digitalWrite(ledPin2, HIGH);
      break;
    }
    break;
  case LEDTRICOLOR:
    switch(state) {
    case 0:
      digitalWrite(ledPin3, LOW);
      digitalWrite(ledPin4, LOW);
      break;
    case 1:
      digitalWrite(ledPin3, LOW);
      digitalWrite(ledPin4, HIGH);
      break;
    case 2:
      digitalWrite(ledPin3, HIGH);
      digitalWrite(ledPin4, HIGH);
      break;
    }
  }
}


// sound
void sndPlay(int sound) {
  switch(sound) {
  case 0:               // sound when hitting an obstacle
    sndTone(1275);
    sndTone(1915);
    sndTone(2000);
    break;
  case 1:               // sound when starting
    sndTone(2100);
    sndTone(1500);
    sndTone(1200);
    break;
  case 2:               // random sound
    for(int i = 0; i < 3; i++)
      sndTone(random(1200, 3000));
    break;
  }
}

void sndTone(int tone) {
  for(long i = 0; i < 80000; i += tone * 2) {
    digitalWrite(outSndPin, HIGH);
    delayMicroseconds(tone);
    digitalWrite(outSndPin, LOW);
    delayMicroseconds(tone);
  }
}


// movement
void start() {
  contactCount = 0;
  steerTime = soundTime = millis();
  moving = true;
  servoGo(SPEED);
  sndPlay(1);
  setLed(LEDTRICOLOR, 2);
}

void stop() {
  startTime = millis();
  meanSndCount = 0;
  moving = false;
  servoGo(0);
  setLed(LEDTRICOLOR, 0);
}

void hit() {
  contactCount++;
  servoGo(-SPEED);
  sndPlay(0);
}


// time
long randomTime(long since, long about) {
  return since - (millis() - round(about * random(5, 15) / 10));
}


// main loop
void loop() {
  if(! moving) {
    if(lightTime < millis() - 1000) {
      lightTime = millis();
      long light = 0;
      for(int i = 0; i < 3; i++) {
        pinMode(ledPin5, OUTPUT);
        digitalWrite(ledPin5, HIGH);
        pinMode(ledPin5, INPUT);
        digitalWrite(ledPin5, LOW);
        for(; light < 90000; light++)
          if(digitalRead(ledPin5) == 0)
            break;
      }
      lightCheck = light < 10000;
      
      Serial.println(light);
      if(lightCheck)
        setLed(LEDBICOLOR, 2);
      else
        setLed(LEDBICOLOR, 1);
    }
    
    if(lightCheck && randomTime(startTime, RANDOM_START_SECONDS * 1000L) < 0)
      start();
  }
  else if(contactStep == 0) {
    if(randomTime(soundTime, RANDOM_SOUND_SECONDS * 1000L) < 0) {
      soundTime = millis();
      sndPlay(2);
    }
    
    // check number of hits
    int remaining = BUMPS_TO_STOP - contactCount;
    if(remaining == 0)
      stop();                     // too many hits, stop ;)
    else if(remaining == 1)
      setLed(LEDTRICOLOR, 1);     // turn off one led, we are going to stop next hit
  }
  
  if(! moving) {
    // sound sensor to start
    int snd = analogRead(inSndPin);
    meanSnd[meanSndCount++ % SNDMEAN] = snd;
    if(meanSndCount >= SNDMEAN) {
      float mean = 0;
      for(int i = 0; i < SNDMEAN; i++)
        mean += meanSnd[i];
      mean /= SNDMEAN;
    
      if(lightCheck && (snd < round(mean) - 1))
        start();                  // sound activated start
    }
  }
  else {
    // moving and hitting obstacles
    switch(contactStep) {
    case 0:                       // moving straight
      contact = checkContact();
      if(contact != 0) {
        contactStep = 1;
        contactTime = millis();
        hit();
      }
      else {
        if((! steering) && (randomTime(steerTime, RANDOM_STEER_SECONDS * 1000L) < 0))
          servoSteer(random(LEFT, RIGHT + 1));
      }
      break;
    case 1:                       // moving backwards after hit
      if(contactTime < millis() - BACKWARD_MSECONDS) {
        contactStep = 2;
        servoRotate(contact);
      }
      break;
    case 2:                       // rotating after moving backwards
      if(contactTime < millis() - (BACKWARD_MSECONDS + ROTATION_MSECONDS + random(0, 200))) {
        steerTime = millis();
        contactStep = 0;
        servoGo(SPEED);
      }
    }
  }
  
  // go on accelerating
  servoContinue();
}

In this code there is some modifications to do, because now I use ir sensor instead of winkers, and the piezo sound output would be the speech output. There I have to put serial outputs and in processing “if” for serial inputs?

#8

In the Arduino IDE, select File + Examples + Communication + SerialCallResponse (or SerialCallResponseASCII).

Both examples have complete Arduino and Processing code to get the Arduino to communicate with a Processing sketch.

From the Arduino sketch, you can see how to write data to the serial port.

From the Processing sketch, you can see how to read that data, how to do something based on what was read, and how to reply to the Arduino (which may not be necessary for you to do).

There are examples available in Processing, too. There are examples that show how to display a picture in the window (3D + Image + Explode) and to play an mp3 file (Libraries + Minim (Sound)).

#9

Ok tonight I will try something. I hope I can help you too!! Thank you very much, tomorrow come and see what I produce!!

#10

Hi! I am trying to build an example for noobs like me. The example communication - serial that sends an "A" is not simple. For this reason I am writing a program that writes "Hello world" in processing when a button is pressed:

I have a problem: -If I open processing and arduino in the same time arduino program says that the comport is busy. If I open processing after arduino, the board stop to send "A".

Here it is the button/"hello world" code under costruction

#define button  2   
void setup()
 {
   // start serial port at 9600 bps:
   Serial.begin(9600);
   pinMode(2, INPUT);   // digital sensor is on digital pin 2
 }

void loop()
 {
  Contact();          
   }
 }

void Contact() {
 if(digitalRead(button) == HIGH) {
     Serial.print('A', BYTE); 
     delay(300);
  }
 }

For processing part I am far from success

import processing.serial.*;

int bgcolor;                       // Background color
int fgcolor;                       // Fill color
Serial myPort;                       // The serial port

control if arrives "A", if arrives println('hello world');
#11

You are not trying to communicate between the Processing application and the Arduino IDE. So, don't open the Serial Monitor window from the Arduino IDE. Doing that binds to the COM port that Processing is trying to use to talk to the Arduino.

With the Processing IDE running, but not running a program, use the Arduino IDE to upload the sketch to the Arduino.

Open the Serial Monitor window, and push the button on the Arduino. You should see some data appear in the serial monitor window. It may not be an A. The Serial.print function sends characters as characters, unless told otherwise. You are telling it to send the character's ASCII value as a byte.

I'm not sure what affect that will have, but it's completely unnecessary. If you want to send a character, and receive and understand it as a character, don't use the BYTE option.

When you know that the Arduino is sending recognizable data to the serial port, as seen in the serial monitor window, close the serial monitor window.

Then, select the SerialCallResponseASCII processing example, and run it.

You should see the same data being sent from the Arduino when you press the button.

Let us know how it goes.

#12

i find this very useful library: http://processing.org/discourse/yabb2/YaBB.pl?num=1204335245/10 very simple to use...

now I am thinking how recall a quotation if sensor readings high do you have some suggestions?

#13

Ok it WORKS! here it is a video of the first steps http://www.youtube.com/watch?v=My_F8sGu9Ac it is only an hello world. I send via serial a A to say Oh my god i am alive why? a B to say let's go and understand something and a C to say f___ing obstacle :-X in arduino I used a simple if statement to get the right message to say. Now I would like to send more complex messages like a value (like a "T" and a numeric value like "21" to call a speech like "temperature is 21"). Can you help me!?!? ;)

I would like to create a speech generator based on inputs like roborealm, ir, ultrasonic range finders, temp sensor, tilt ecc...

#14

Can you help me!?!?

The code that is reading the current commands can be easily expanded to handle more complex commands. There are some libraries, like Messenger, that facilitate this. But, doing it yourself isn’t difficult. Post the code you have now, so we can suggest how it might be extended.

#15