Animatronic Parrot (mostly audio questions... turned out to be a lot of other stuff too!)

Projects Discussion and Showcase Interactive Art
1

Hi all, first time poster so I hope this is in the right place & my questions make sense. I will do my best to explain my goals, current position, followed by the questions I need help with. Sorry if its a bit long winded but I figured I should get as much information into the post as possible. Also I was not sure if I should post in this "Interactive art" forum or the "robotics" forum, but as you read my design goal my choice should become clear..... basically I do not need this to be able to think/interact with environment, it more so a remote controlled prop.

Design goal:
Like many people I enjoy fancy dress when out at festivals/nights out, I am working on a few different wearable animatronics for this reason. This one is for an animatronic parrot to sit on my arm/shoulder which will sit in "idle" mode where it will do random movements/sounds at random intervals, but also respond to a remote control when needed. It will need to be completely self contained as in all electronics & power onboard, the remote will obviously be a separate module that I can operate secretly in my pocket.

Plan / current status:
I have ripped apart a "FurReal Friends Squawkers McCaw" toy parrot & binned the onboard controller. I have extended the internal wiring to a breadboard so I can easily connect to the input/outputs which I am happy with controlling. Its movements are done through two 6v DC hobby motors connected to a series of cams which produce different motions depending on which direction the motor operates. So I currently have access / working of the following things;

  • Head motor: Direction 1 open's mouth, spring loaded closed when motor is stopped. Direction 2 cycles eyes open/closed.

  • Body motor: Both directions operate a series of cams which cause the body to move, wings to flap, head/neck to rotate.

  • Speaker - 0.5w 4Ohm speaker for sound

  • The following limit switch's / buttons, all 2 wire open/closed switches;

  • Tongue press switch - the original toy had a cracker which you could put in its mouth to depress the tongue & trigger an "eating" sequence (probably will not be used, but nice to have the option)
    Eyes open limit switch - Limit switch for when the eyes are in the most open state.
    Eyes closed limit switch - same as above
    Leg limit switch - connected to a cam on the body motor which is activated on various points during the motors sequence. (again probably will not use, but it is something I have access too)

  • There are a few other sensors on board connected through some funny ways which I do not think will be worth the effort to get working as I probably will not use them anyway.

Breakdown of plan/goal:
In the most basic sense I need to synchronize mouth movement to mp3 sounds played along with random body movements.
Expanding on that idea, I will run the animatronic in two basic modes;

  • Idle: Store a selection of mp3's in an "idle" folder. At random intervals either play a random mp3/sync mouth or activate body motor for random time.
  • Remote trigger: I have an 18650 integrated ESP8266 & 4x4 button matrix as a remote control. Split mp3's into 16 folders. Using ESP-NOW play random mp3 from selected 1-16 folder.

Intermission - unplanned question 1
I had not really thought to much about doing a project log, but I realise with how much I have already typed it might be an idea. So would anyone else be interested in me turning this into a log with photos, videos, updates that may help if you decide to do a similar conversion??
Okay, back to it!

Current status:
First up I am a bit of a newbie with all of this, so my design practice is very messy & inefficient. I have an ESP8266 + motor shield driving the motors which I have working. I can read the output from the eye limit switch's but have not implemented them yet as I have been focused on syncing mouth/mp3 movement.
I really should have done a circuit diagram of the next part, but will do that later to upload & maybe a video of current working. I have a BT201 module playing mp3's from an SD card, this is currently using onboard controls but with software implementation I should be able to control using AT commands on serial connection (please see in my questions section why I am thinking of changing). The BT201 module has a mono speaker connections (combines stereo to mono) and also a stereo headphone jack. BOTH of these outputs function at the same time, I was able to exploit this to allow me music/signing synchronising -- I edited mp3's so that all "vocals" were on the left channel & all "music" was on the right channel. Both play through the mono speaker output so you can hear the vocals & music. However from the headphone Jack I have connected the left "vocal" output to the ESP's analogue input which I read to trigger the mouth motor.

Current thoughts / roadmap / changes etc
Really was not sure what to title the heading as, but this is kind of a link between where I currently stand & why I am thinking about changing the current plan (which will link into my questions in the last segment)

  • Eye control: I can read the eye limit switches & control the motor, however I have not implemented code to synchronise the two.
  • Body movement: I can control the motor to make it run through the predefined series of motions, I can read the body limit switch, however I have not implemented code to synchronise the two.
  • Tongue sensor: I can read the input, but have not implemented any code for it yet as I just do not know what I would use it for currently.
  • Mouth/sound: This is the messy part. I have implemented code to trigger the mouth movement if the audio received on the analogue input exceeds a certain level. Watching the parrot sing to songs is most entertaining (I may be 40 but watching a parrot karaoke to "Walking on sunshine" with the voice of Gilbert Gottfreid is making me giggle like a 4 year old!!!!!) So this splitting of mp3's into left channel vocal, right channel backing track, has GOT to stay!!! With that being said there are a few issues I have run into using this BT201 sound module, which is the major cause for considering changing what I have currently.

Current issues / why I am thinking of changing

So I am thinking about moving this project over to an I2S based system instead, but I have never tackled any project using this (as stated I am a complete newbie to this!). Not to sure how I am going to explain/format questions so please bare with me;

  • 1: The physical size of ESP8266, Motor shield, BT201 are at the very limit of what I can fit into the parrot. Even that is going to need some heavy modification which I am happy to do, but would rather avoid. Either way there will be some heavy modification to add extra space for the battery pack. The location of the original controller board really does not offer much in the way of usable space as both the circuit board & the parrots body was built around this, there for I am planning to use the area of the battery pack to hold the electronics & then add a hidden battery compartment elsewhere.
  • 2: The ESP8266 motor shield only allows access to 8 digital I/O's & 1 analogue input. 4 of the digital I/O's are linked to the motors. So if I continue to use this setup I am limited on what I can do.
  • 3: Only able to read analogue on one channel. As stated, the music/signing is very entertaining so will be something I plan to keep, however both the mouth movement & random body movement are triggered by the vocal input, this means there are times where just the backing track are playing & the parrot sits dead still. It would be nice to be able to read both Vocal/Music channels so I can sync mouth to vocals but also still have random body movement if music is playing. I think that I could query the BT201 to see if playing = true & use that to trigger the random movements, but I am concerned about sending to much traffic across serial to/from the BT201. Not to mention that I also need to make the serial connection in the first place which will cost 2 of my I/O ports and also having to try & make sense of the AT commands around the internet.

The new plan / the questions
So this is where I will outline what I am thinking about doing different & where I would really like some insight as to weather this is a sensible plan. As I said I am new to this, I am not asking anybody to do my work / solve my problems....... Its more to confirm or correct me if I am going about it the wrong way. As we all know learning this stuff takes a lot of time & I would rather someone tell me if I am wasting my time looking at something which will not work when I could be looking at something else which would do the job.
So..... I am thinking about putting all of this into the spares box & kind of starting again from scratch.

  • ESP: I have several different ESP32 boards laying about which are much smaller than the current setup & have access to many more of the I/O's. So changing to one of those seems like a sensible idea. Also I think I should stick to ESP MCU's as it will allow the use of ESP-NOW for the remote control.

  • Motor shield: I went for the 8266 & motor shield combo as I had thought it would be the easiest solution. After playing about with it I now realise that it is a limitation both in size & connections. I only need to drive two DC motors & I have some MX1508 H-bridge drivers laying about. These are a fraction of the size so again it seems like a sensible idea to change over to using these instead.

  • Audio section / where my real questions are: The BT201 board is pretty large, the footprint is the same as the motor shield, although 1/3 the thickness. Connecting to this board through serial seems a little be janky & limited, this is why I am thinking about swapping over to I2S instead but have no knowledge at all about it. So today I have been having a little read/look at the following library & modules. The following questions are where I would like some confirmation or guidance before I spend out any more cash on things which will end up living in my spares box!!
    So this is the library Arduino Audio Tools
    I2S Amplifier Max98357
    SD card reader SD card reader

  • Q1: Would that SD card reader be suitable? Either way having a small reader module would help with space limitations but also mean I could place it in a more suitable area for access when I need to update the files.

  • Q2: I believe that selecting a folder & playing a random mp3 would be easier to code using this SD reader than trying to code AT serial commands for the BT201 module, or am I thinking wrong about that?

  • Q3: Am I right in thinking that using the "VolumeOutput class" of the library I should be able to measure the output of both left/right channels, allowing me to synchronise mouth movement to the left channel & random body movement to the right channel?

  • Q3A: This is more a thought as I do not really need it but might be nice to look into later on down the line when everything else is complete..... Using this setup would it be possible to count beats on the right channel? kind of like you see with disco lights etc? this would allow me to sync the random body movements to the timing.

Final thoughts
Thank you very much if you have managed to get through that essay! I had not planned for it to be that long but here we are. I am thinking now that I will turn this into a combination "help me I'm lost" / "this is what I have learned" project log. I intend to condense all of this down into something more palatable, but will leave as is for the moment as it has already been incredibly useful to myself just getting all of this into words!! So rather than rushing off now & getting myself overwhelmed with all the possibilities, I think I will spend the rest of the evening grabbing some photos/videos of the current project as well as working on some circuit diagrams so that the following post will be a little less wordy & a little more interesting!!

Thank you again!
Pearl

1 Like
2

Just a quick update to add some clickable interest to the post..... its 1am here so wont be doing any fancy circuit diagrams etc tonight, but have a couple of videos of the current operation of the parrot. Also to note, these are all "dumb" clips..... as in they are only activated by pressing play on the BT201, there is currently no communication between the ESP8266 & BT201 other than reading the analogue audio signal.

The first video is an "introduction" to the new youtube channel explaining what it is all about. However for here it is an example of syncing the motors to the sound being played. As a side note, the volume of the output speaker is turned down slightly to not annoying housemates in the middle of the night, also the motor sound is not really that loud in person, not helped by the fact it is sat on a wooden board on a kitchen surface!!

Fredd13 Introduction

Much like the first video, the second is just showing it synced to random sounds that I have clipped from birds on youtube with a few of speaking that I have added.

Random Sounds

The final two are to show what was meant by the singing mode. Left channel contains the vocals & is read on the analogue input to animate the motors. The right channel contains the backing track. These also show how out of place it looks to have the music playing but no animation of the body which is why I am thinking of changing to a system where I can monitor both channels, left for mouth sync, right for body movement / dancing.

Edit: Also in the first video (Man! I feel like a parrot) you can see a small issue at the end where the mouth misses one of the vocal sounds. This is an issue where I have been trying to set a cut off value when reading the audio input because of the amount of noise on the line I just cant seem to get rid of, also another reason why I am thinking about going I2S and analysing the digital mp3 levels as its playing rather than reading an analogue audio input. /edit

Man! I feel like a parrot!
Fredd13's stayin' Alive!

I hope this adds a little interest to the thread & that it helps make my idea's a little easier to understand.

3

i am still looking for the part that describes what parts you have tried and experimented with.

4

Hi Paul, thanks for the response :slight_smile:

The ESP8266 & motor shield are here ESP + Motor shield
And the mp3 player was this one here BT201

The above is what is being used on the videos I linked to.

However since then I have been doing a little more searching and the I2S solution with the previously mentioned MAX98357 & separate SD card reader looks like it could be very promising so I have gone ahead & ordered those parts..... if it fails then atleast I will have some more useful parts to go in my spares box!

Also in my spares box which I plan to use for this projects are some ESP32-S2 mini or ESP32 Dev board 38 pin

I believe that the S2-Minis are only single core, although still 240mhz, obviously would be helpful with the physical size, however if I am going to be doing realtime processing of audio then I think the larger dual core ESP might be a better option?

Also incase needed, these [L298N based MAX1508] (sorry had to remove the link as I am only allowed to post 4 links per post as new user!) is the H-bridge driver I am planning to swap to for the DC motors.

The new modules should be arriving within the next 48 hours so should be able to have an experiment with them then.

Again, thank you for the response & I hope that I have posted the relevant information back.

5

More words and no information on what components you, personally, have written programs for and gotten to work.

6

Lot to unpack here, but following this thread as I'm way more of a newb than you and have a project requiring sound.movement synchronization. Question (that doesn't really require an answer,) have you considered servos instead of motors for the movement? Micro's are pretty small.

1 Like
7

In the videos I am using an ESP8266 mounted on to a motor shield, the only connections made to the BT201 audio module are power to the board & a returning analogue signal from the headphone jack. The BT201 module linked above has onboard PLAY/PAUSE/NEXT/PREV buttons which I am using for testing.
I have not posted schematics or code I personally wrote for this because both the physical size along with limited I/O's, after testing, make it completely unsuitable for my needs so it a dead end as far as I am concerned. It was enough to be able to tap into the parrots motors/switches to test if it was possible to interact with them.
I have also messed around with a DY-SV17F which did not have anywhere near enough onboard memory, which is why I moved to modules with SD-Cards. I tried an DY-SV5W however you can not use both the headphone jack & speaker connections simultaneously.

So those are some of the reasons why that part of my project is dead in the water & now living in my spares box for another project one day... I do not plan to do any more with those components nor write programs for them.

That is why I was asking if my thoughts about moving to an I2S system would be a worthwhile idea. I am happy with the snippets of code I have written to control the motors and will be simple enough to port across, all I need is a signal to trigger them, that is why I was questioning reading left/right channel volumes of an mp3 in an I2S system, that way I can use the peaks to trigger the motion code.

So as of right now, to get to a minimum point of my goal, components are as follows:

ESP32 MCU - I have a couple of different versions, will experiment with to choose most suitable
MAX1508 H-bridge - have not used yet as I do not have the other components
2x 6V DC hobby motors - Built into the parrot & working fine
MAX98357 I2S amplifier - have not tested yet as not arrived
3.3v SPI SD card reader - have not tested yet as not arrived
4Ohm 2W speaker - now built into the parrot body & working fine

Code - have not written yet as components have not arrived
Circuit diagram - have not illustrated yet as components have not arrived

Once parts have arrived I can put a circuit together & start testing the Arduino Audio Tools library for retrieving the volume levels of left/right channel as well as playing mp3's from the SD card to the MAX98357 I2S amplifier. At that point I will be able to start sharing used component lists, programs I have personally written and circuit diagrams.

Hi Rocknbil, yup sorry for all the wording, just trying to explain my situation to the best of my ability! I hope you will be able to find some useful information or at least personal questions/ideas/etc that you may not have thought of.
As for the servo's, that really is just down to the way the original parrot toy is built, they just are not a suitable solution because of the series of rotating cams attached to the hobby motors. However if I was starting this completely from scratch, 3d printing the parrot etc then I would definitely have gone for servo's instead. Honestly they would be a much better solution & allow a lot more control/variation over what I have now but to convert this toy to servo would require so much reworking of the internals that it would be all around easier to start from scratch with a 3d printed body. If I knew that the parrots electronics were going to be so damaged when I bought it from ebay, then I would have gone scratch building instead..... Since the seller is being funny about stuff & I cant get my cash back, then I wanted it to go to at least some use.

/edit

Although I as I have stated the first part of my project is now dead & will not be experimented with any further, I have had a quick clean up of the code it was running & will attach below. Please note that it is not optimal for two main reasons:
1: I just don't have great skill's with coding yet, this worked well enough for what was needed in the initial tests / videos
2: there are lines in there for when I was setting up for other tests which I did not complete but planned to at a later date before declaring this part of the project dead.

When the new parts arrive & I can begin testing/experimenting, I will transfer some of this & bin other parts.

#include <FastLED.h>

#define LED_PIN     D5
#define NUM_LEDS    1
#define BRIGHTNESS  255
#define LED_TYPE    WS2812
#define COLOR_ORDER RGB
CRGB leds[NUM_LEDS];

#define UPDATES_PER_SECOND 100

const int analogInPin = A0;  // ESP8266 Analog Pin ADC0 = A0
const int pwmMotorA = D1;
const int pwmMotorB = D2;
const int dirMotorA = D4;
const int dirMotorB = D3;
const int LEDPin = 16; //onboard LED, not WS2812
const int legPin = D6;
const int eyePin1 = D7;
const int eyePin2 = D8;

//timing for mouth/body motors
int mouthMillis = 0;
int mouthDelay = 25;
int danceMillis = 0;
int danceDelay = 100;
int danceRev = 0;

//integers for storing analogue input values
int sensorCal = 0;
float sensorPer = 0;
int sensorValue = 0;

//integers for storing limit switch values
int legSensor = 0;
int eyeSensor1 = 0;
int eyeSensor2 = 0;

//for storing the speaker/body WS2812 brightness levels
int glow = 255;

//for determining if in "idle" mode
int timeIdle = 0;
bool isIdle = true;


void setup() {

  //initialize serial communication at 115200
  Serial.begin(115200);

  pinMode(LEDPin, OUTPUT);
  pinMode(pwmMotorA , OUTPUT);
  pinMode(pwmMotorB, OUTPUT);
  pinMode(dirMotorA, OUTPUT);
  pinMode(dirMotorB, OUTPUT);
  pinMode(legPin, INPUT_PULLUP);
  pinMode(eyePin1, INPUT_PULLUP);
  pinMode(eyePin2, INPUT_PULLUP);

  //attach WS812 LED
  FastLED.addLeds<WS2812, LED_PIN, GRB>(leds, NUM_LEDS);
  fill_solid(leds, NUM_LEDS, CRGB::Black);
  FastLED.show();
 


  digitalWrite(LEDPin, LOW);

  delay(5000); // BT201 board plays a startup sound when powered on, wait for this to pass before calibrating


  // Calibrate levels on analogue input & output the values to serial
  for (int cal_int = 0; cal_int < 1000 ; cal_int ++){
    sensorCal += analogRead(analogInPin);
    delay(3);
  }

  sensorCal /= 1000;
  

  Serial.print("Sensor Calibration = ");
  Serial.println(sensorCal);
  Serial.print("Sensor Percentage = ");
  Serial.println(sensorPer);
  delay(1000);

// flash LED's to confirm calibration & moving into main loop

  digitalWrite(LEDPin, HIGH);
  fill_solid(leds, NUM_LEDS, CRGB::Black);
  FastLED.show();

  delay(250);
    digitalWrite(LEDPin, LOW);
      fill_solid(leds, NUM_LEDS, CRGB::Red);
  FastLED.show();

  delay(250);
      digitalWrite(LEDPin, HIGH);
        fill_solid(leds, NUM_LEDS, CRGB::Black);
  FastLED.show();

  delay(250);
    digitalWrite(LEDPin, LOW);
      fill_solid(leds, NUM_LEDS, CRGB::Green);
  FastLED.show();

  delay(250);
  digitalWrite(LEDPin, HIGH);
  fill_solid(leds, NUM_LEDS, CRGB::Black);
  FastLED.show();
}

void loop() {



  // read the analog in value & convert to percentage
  sensorValue = analogRead(analogInPin);
  sensorPer = ((float)sensorValue / (float)sensorCal);

  delay(1);
  timeIdle += 1;

  // Fade the speaker LED mounted in body
  if (glow > 0){glow -= 1;}
  fill_solid(leds, NUM_LEDS, CRGB::DarkRed);

  FastLED.setBrightness(glow);
  FastLED.show();

  
  // set if enough time has passed to enter "idle" mode
  if (timeIdle <= 5000){isIdle = false;}
  if (timeIdle >= 5001){isIdle = true;}
 

//Open mouth & increase brightness of speaker LED if analogue input passes threshold, also flash the onboard LED to confirm threshold value has been reached 
  if(sensorPer > (2)){
    digitalWrite(LEDPin, LOW);
    mouthMillis = millis();
    digitalWrite(dirMotorA, HIGH);
    digitalWrite(pwmMotorA, HIGH);
    if(glow <= 150){glow += 10;}
    if(glow >= 150){glow += 100;}
    timeIdle = 0;
  }

//activate body/dance motor if threshold has been reached
  if(sensorPer > (4)){
    if (danceRev <= 1){digitalWrite(dirMotorB, HIGH);}
    if (danceRev >= 2){digitalWrite(dirMotorB, LOW);}
  
    digitalWrite(pwmMotorB, HIGH);
    danceMillis = millis();
    danceDelay = random(500);
  }
      
//close mouth, turn off onboard LED, fade speaker LED, if "mouthDelay" time has passed
    if ((millis() - mouthMillis) >= random(10, mouthDelay)){digitalWrite(pwmMotorA, LOW);
    digitalWrite(LEDPin, HIGH);
    if(glow >= 100){glow -= 50;}
    }

//stop body movement & select random motor direction after "danceDelay" has passed
    if ((millis() - danceMillis) >= danceDelay){digitalWrite(pwmMotorB, LOW);
    danceRev = random(0,3);

    } 

}
8

Okay, I guess this will be the first real post on the project log as such because I am starting it again with the outlined goal in mind. As you read through, I would please like to ask that you take the following under consideration:

1: Because all of my previous projects (including outside of electronics) have always been for personal use & never really shared, I want to make it clear that this is the first time I have ever done any kind of project log. Do not expect it to be easy reading etc as there will be a lot of words/personal thoughts to give an insight into what I want to achieve & how as a complete newbie I am thinking about achieving those goals.

2: I am a complete newbie with coding so please bare that in mind. Most of the code will be non-optimal, butchered together from examples, done in a back to front way etc. I appreciate that many of you will find these things trivial, however for me many of them will be big achievements! Lets just remember we all started somewhere.

3: Finally, I have my goal in my head & many parts of that have already been stated in the previous couple of posts. There is a chance that some of those goals/ideas may get changed or modified along the way to tie in with my skill set.

I appreciate advice & constructive criticism, but lets try to keep the plain negative things to ourselves, after all makers are supposed to support & encourage each other!

The first update

What I thought was a huge achievement for today & what I hope will help with testing during the rest of the project was getting an ESP-NOW remote control to work! Its nothing fancy at all, just a 4x4 button matrix module soldered onto the side of an ESP8266 with onboard 18650 battery, it looks rough at the moment but I will design some kind of enclosure to 3d print over the next few days. I really like that there is an onboard charging feature via micro USB which will certainly make things easier when I am stuck in a tent trying to keep the batteries full! So here we have a photo of its current state & I will discuss the code / thoughts afterwards.

Parrot Remote
Parrot Remote1536×1024 314 KB

So the following code is kind of a butchered variation of the keypad.h example & an ESP-Now example I found online. Looking at the photo of the matrix it would make sense that (S1, S2, S3, S4) was the top row, but no matter how I declared my pins..... the best I could get was for it to be the right most column, which would make sense if the header was at the top of the matrix. In the end it was easier to follow that assumption & just rename the buttons in the code.
In order for me to send commands I thought I would use a simple 3 digit integer comprised of "Row, Column, Press type". Currently the code only implements one press type but later on as I learn more I would like there to be at least short/long press. My thinking being is that my group of friends all have a sense of dark humour so many of the responses will be tailored around that, however being at festivals etc, I also have to think about the children, so even though all of the buttons will cause the same "type" of responses I am thinking about splitting them into short press = child friendly / long press = not child friendly. Simple example of commands being sent;

Row 2, Column 4, short press = 241
Row 3, Column 1, long press = 312

I think that should make it easy to read/code for each command without having to refer to "which code is that button" sheet etc and also make it fairly easy to decode on the other end as all the commands will be following the same 3 digit system.

So here is the code I am currently running on the controller:


#include <ESP8266WiFi.h>
#include <espnow.h>
#include <Keypad.h>


const byte ROWS = 4; //four rows
const byte COLS = 4; //four columns
//define the symbols on the buttons of the keypads
//Arranged strangely because my keypad is orientated 90 degree to the right, right column = top row
char hexaKeys[ROWS][COLS] = {
  {'C','8','4','0'},
  {'D','9','5','1'},
  {'E','A','6','2'},
  {'F','B','7','3'}
};
byte rowPins[ROWS] = {D4, D3, D2, D1}; //connect to the row pinouts of the keypad
byte colPins[COLS] = {D8, D7, D6, D5}; //connect to the column pinouts of the keypad

//initialize an instance of class NewKeypad
Keypad customKeypad = Keypad( makeKeymap(hexaKeys), rowPins, colPins, ROWS, COLS); 


// REPLACE WITH RECEIVER MAC Address
uint8_t broadcastAddress[] = {0x60, 0x01, 0x94, 0x4A, 0x8D, 0xCB};

// Structure example to send data
// Must match the receiver structure
typedef struct struct_message {
  int a;
} struct_message;

// Create a struct_message called myData
struct_message myData;

unsigned long lastTime = 0;  
unsigned long timerDelay = 2000;  // send readings timer

int LEDpin = 16;
int command = 0; //keypad command to send





// Callback when data is sent
void OnDataSent(uint8_t *mac_addr, uint8_t sendStatus) {
  Serial.print("Last Packet Send Status: ");
  if (sendStatus == 0){
    Serial.println("Delivery success");

      }

  else{
    Serial.println("Delivery fail");

      }
}
 




void setup() {

  pinMode(LEDpin, OUTPUT);
  digitalWrite(LEDpin, LOW);

  // Init Serial Monitor
  Serial.begin(115200);
  delay(1000);
  Serial.println("Serial started");
  digitalWrite(LEDpin, HIGH);
 
  // Set device as a Wi-Fi Station
  WiFi.mode(WIFI_STA);

  // Init ESP-NOW
  if (esp_now_init() != 0) {
    Serial.println("Error initializing ESP-NOW");
    return;
  }

  // Once ESPNow is successfully Init, we will register for Send CB to
  // get the status of Trasnmitted packet
  esp_now_set_self_role(ESP_NOW_ROLE_CONTROLLER);
  esp_now_register_send_cb(OnDataSent);
  
  // Register peer
  esp_now_add_peer(broadcastAddress, ESP_NOW_ROLE_SLAVE, 1, NULL, 0);
}
 
void loop() {
  digitalWrite(LEDpin, HIGH);

char customKey = customKeypad.getKey();
  
  if (customKey){
      
  if (customKey == '0'){command = 111;}
  if (customKey == '1'){command = 121;}
  if (customKey == '2'){command = 131;}
  if (customKey == '3'){command = 141;}
  if (customKey == '4'){command = 211;}
  if (customKey == '5'){command = 221;}
  if (customKey == '6'){command = 231;}
  if (customKey == '7'){command = 241;}
  if (customKey == '8'){command = 311;}
  if (customKey == '9'){command = 321;}
  if (customKey == 'A'){command = 331;}
  if (customKey == 'B'){command = 341;}
  if (customKey == 'C'){command = 411;}
  if (customKey == 'D'){command = 421;}
  if (customKey == 'E'){command = 431;}
  if (customKey == 'F'){command = 441;}

  Serial.println(command);
  myData.a = command;
      // Send message via ESP-NOW
  esp_now_send(broadcastAddress, (uint8_t *) &myData, sizeof(myData));
  }
  
}

As for the receiver in the parrot itself...... I have stripped everything back to just an NodeMCU ESP-12e module for testing communication. The final will be running on some kind of ESP32 module which I have not decided on yet, but wanted to at least get a basic ESP-NOW system working that I can port across later on. Currently all that it does is listen for a command & then print that integer to serial, while also printing the seconds since last receiving a command. Pretty boring for an update I know, but for me this was the first big step to take in this project and what I thought was going to be one of the hardest parts!!! So here is the code I am running on the receiver which is a modified example code that currently does the job!



#include <ESP8266WiFi.h>
#include <espnow.h>

int command = 0; //command recieved from remote
int lastMessage = 0;  //seconds since last message

// Structure example to receive data
typedef struct struct_message {
  int a;
} struct_message;

// Create a struct_message called myData
struct_message myData;

// Callback function that will be executed when data is received
void OnDataRecv(uint8_t * mac, uint8_t *incomingData, uint8_t len) {
  memcpy(&myData, incomingData, sizeof(myData));
  Serial.print("Command: ");
  Serial.println(myData.a);
  Serial.println();
  command = int(myData.a);
  lastMessage = 0;
}
 
void setup() {
  // Initialize Serial Monitor
  Serial.begin(115200);
  delay(2500);
  Serial.println("Serial Running");
  
  // Set device as a Wi-Fi Station
  WiFi.mode(WIFI_STA);

  // Init ESP-NOW
  if (esp_now_init() != 0) {
    Serial.println("Error initializing ESP-NOW");
    return;
  }
  
  esp_now_set_self_role(ESP_NOW_ROLE_SLAVE);
  esp_now_register_recv_cb(OnDataRecv);
}

void loop() {
  delay(1000);
  lastMessage += 1;
  Serial.print("Sec's since last message = ");
  Serial.println(lastMessage);
}

Final thoughts for the day

Again, this is probably pretty trivial to many of you, but I am quite proud that I managed to get this working the way it currently is. As this is going to be a mobile animatronic it will not always be possible to connect up to serial for error/debug messages, with that in mind I have added a single WS2812 LED into the chest/speaker area of the parrot which will serve two functions: 1, to add a glowing effect while it is talking, I would like to try & imitate something a little like the warm glow of an old vacuum tube that varies on how much is being said. 2, for functionality I would like to write a series of colours/pulses which would work as a very rudimentary "error code" when out in the field...... No I will not be in a position to plug into a workstation & modify code while I am there, but at least I can make a note of it for when I get back home.

With all that in mind & that I already have the WS2812 in place / wired up, I guess the next logical step will be learning/experimenting with getting the received commands to simply change the LED colour depending on what button is pressed. I know it would be trivial to have if/then/update_led routine in the main loop. However I believe that later on down the road I will have to use "void functions" to achieve my goal..... which again is all new to me, so I guess that is what I will be looking into next.

Wish me luck!!!!!

Goals for next update

Minimum: Have useful communication between transmitter & receiver to call a function changing colour of an LED depending on the button pressed.

Would be nice: Also Change the receiver to an ESP32, hook up an L298N H-bridge & have motors on/off forward/reverse command functions working to allow further testing.

Bonus: Get all limit switches hooked up so I can start probing the way they trigger at certain times in the motors movements.

9

Not that hard. But you might use UDP and send data packages continously. That's a lot easier to handle than a data stream that can get packetized at any position.

1 Like
10

Thank you for the advice @zwieblum. I had a quick look on google at UDP and to my understanding (which could be completely wrong!) it looks like UDP would only be used on a normal WiFi connection and not with an ESP-NOW network?
The current ESP-NOW code I am running (previous post) does contain delivery/reception confirmation which I am making an assumption would detect if there were an issue with broken packets?
I am also making another huge assumption that sending data packages continuously would increase the power usage?? As both the remote control & parrot will be running on batteries I would like to keep power drain to a minimum.

Update 1(A): Release the magic blue smoke!

So for the second time on this project I have blown up a BT201 mp3 module! oopsie! I know the component that was at fault in a 1k linear potentiometer. The first time this happened was just bad planning by me, I assumed that I could just output from the BT201 module, through the pot & into the speaker..... just into one pin of the pot & out of the other (I cant remember which pins it was, but it did work....... for a while anyways!)

The second time I wired it like shown in a youtube video following this picture:

speaker pot
speaker pot1280×720 77.6 KB

Just assume that the headphone jack used on the picture was from the speaker connection of the BT201 module. Has been working fine for a few days, but just an hour ago it stopped again. There had been a little crackling when changing volume so not sure if it was a duff pot or if my soldering was off.

Either way, twice now I've managed to break these boards!

Why this way? Why the change?

The BT201 module outputs sound simultaneously through both the mono speaker & stereo headphone jack. This was a perfect combination allowing me to play both audio channels trough the speaker for the sound output I needed, while also allowing me to connect the left channel from the headphone jack to the Analogue input of the ESP8266 allowing me to read the vocal signals to sync the mouth movement too. Everything worked fine apart from one aspect..... The volume needed to be at maximum to be able to read enough variation on the analogue signal to determine what was sound vs noise. The BT201 has onboard volume control but that affects both the headphone/speaker at the same time, making the speaker very loud!!! Not so good when you are doing most of your experimenting at night while other people in the house are trying to sleep. Hence adding the pot inline with the speaker for independent control.

So lessons learned from that setup are that reading the analogue signal this way contained a lot of noise. I followed the circuit diagram below to build a filter which did help with the issue, however the volume still needed to be max. Maybe it was the wrong design to use? I don't know, just trying my best to make sense of things as I go along!

audio_1

So with all that being said & done, Where do we stand now?

Playful Technology on github has also done a lot of work converting one of these toys to Arduino. I have been working over their code & spliced it together with some ESP-NOW code I had been playing with, allowing me to call certain movement functions from the buttons of the remote. It had been working relatively fine most of the afternoon however I think the head motor is sticking a little bit so need to take the parrot apart to check for anything catching inside...... but the more I think of it, the more I am thinking about looking to see if I can cut it apart to replace the single DC motor for eyelid/mouth movement with a pair so micro servo's which will just make control so much easier without needing to worry about the way they have implemented their limit switches... well to be honest its more the eye side of it. The mouth is easy apply voltage to open mouth, when voltage stops a spring snaps the mouth closed. If you run that same motor in reverse it cycles through eyelid movements which you have to stop at certain limit points..... would be so much easier to do with a servo!!!

Anyways, I need to be sensible & have a walk away from that side of it for a moment while I still have my calm! I guess that tonight / tomorrow will probably be having a little bit of a play with getting i2s sound rather than getting myself worked up too much over the movement control.

Anyone who has I2S sound knowledge:

I have put together this flow chart to try & better explain the way I am thinking about syncing movement to sound. I am not after anyone to do the coding for me, but if you have done something similar could you confirm/correct me if I am planning this the right/wrong way?

audio processing flowchart
audio processing flowchart1093×807 96 KB

Again, sorry for being all a bit wordy but I figure that if I an not show in plain code what I am trying to do, then giving my thoughts / explanations are the next best thing.

Thank you

11

Yes. Depending on your setup this is good or bad. In my case I have 250 animatronic devices that need realtime syncronised data, so a p2p connection with handshaking is troublesome.

Ath the sound thingie: if you have the storge, then decode MP3 to raw data that you can send over I2S directly. It might turn out that decoding the MP3 in realtime is problematic (as is playing arbitrary parts in the file if you use triggers for different sounds)

1 Like
12

Thank you again for your advice @zwieblum :slight_smile:
Wow that is a lot of animatronics to sync!!! However what I am trying to achieve with my wireless connection is simply to pass a "button press" command. It does not need to be real time synchronizing etc as I plan for everything to be done on the parrot itself.
The reason I was going to use MP3 was to make it easier for creating the sound clips as I am intending on probably several hundred clips across several folders which will then be selected at random. I hope this following example makes sense:

Tell joke:
press button 4 (joke) on remote --> ESP-NOW --> Parrot - button 4 received -> select folder 4 (jokes) -> play random mp3 -> (then do the audio processing / syncing etc)

or, press button 1 (greetings) -------- select folder 1(greetings) ----- random mp3

I guess it should be easy enough to do that with WAV files instead if you think that would make it easier to handle? Also the remote control will never be further than 10ft from the parrot, usually only arms length. Also it will not be sending more than a key press once every 5-10 seconds.

As the project now stands
I think I am going to have to step away from it for a couple of days & admit to myself that I have bitten off more than I can chew!! Apparently just having a solid idea of what I want to achieve & the steps needed are not enough when you do not have the knowledge/experience to implement them :rofl:

So what have I achieved today?
Using the Arduino audio tools library I am now able to to play an mp3 file from an sd card -- control the volume -- output to MAX98357 i2s amplifier. Although this was successful it has also brought up 2 challenges I need to overcome before moving forward.
1: Volume output The 3w amplifier just is not loud enough. I have added a 100k resister between the gain / ground connections of the MAX98357 to boost its gain to maximum. After this I then edited the mp3 file on my PC to boost its levels to maximum without clipping to much.
2: Coding knowledge Reading through various parts of the library it definitely looks like it would be possible to read the left/right channel levels before passing it to volume control handler for output, but I just do not have the knowledge to make sense of it all which is an area I am going to have to work on before moving forward.

Mechanical stuff
I took apart the head of the parrot to give the workings a clean & to see what free space there was for modifying to servo motors. Unfortunately with the way it was built there is no chance of a conversion, it really would be easier to completely redesign & 3d print a servo based head from scratch which defeats the point in trying to save this toy. However it did give me chance to see the limit switches for the eyelids which I think will make things easier when trying to code the movements.

eye limits
eye limits800×854 69.3 KB

There was no easy way to take a photo which shows them in any real detail, but basically as the motor moves the eyelids there is a cam which presses against the limit switches allowing for 3 basic positions:
eyes open = left closed, right open
eyes partially open = left open, right open
eyes closed = left open, right closed

the motor can only turn in one direction so keeps repeating those movements in a cycle (if you reverse the motor direction it engages a different set of gears which control the mouth open/close instead). Looks like I will need to write a truth table & power the motor until one of those 3 conditions are met.

Final thoughts

Today I will leave with the code that is currently working to play a single mp3 file

/**
 * @file streams-sd_mp3-i2s.ino
 * @author Phil Schatzmann
 * @brief decode MP3 file and output it on I2S
 * @version 0.1
 * @date 2021-96-25
 * 
 * @copyright Copyright (c) 2021 
 */


#include <SPI.h>
#include <SD.h>
#include "AudioTools.h"
#include "AudioCodecs/CodecMP3Helix.h"


const int chipSelect=5;
I2SStream i2s; // final output of decoded stream
VolumeStream volume(i2s);
EncodedAudioStream decoder(&volume, new MP3DecoderHelix()); // Decoding stream
StreamCopy copier; 
File audioFile;

void setup(){
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);  

  // setup file
  SD.begin(chipSelect);
  audioFile = SD.open("/pokerface.mp3");

  // setup i2s
  auto config = i2s.defaultConfig(TX_MODE);
  i2s.begin(config);

  // setup I2S based on sampling rate provided by decoder
  decoder.setNotifyAudioChange(i2s);
  decoder.begin();

    // set initial volume
    auto Config = volume.defaultConfig();
  volume.begin(config); // we need to provide the bits_per_sample and channels
  volume.setVolume(1.0);

  // begin copy
  copier.begin(decoder, audioFile);

}

void loop(){
  if (!copier.copy()) {
    stop();
  }
}

Next I am going to have to play around with selecting a file & actual control of the volume. my plan is to edit/code the following part & probably have to move it into its own call back:

31: audioFile = SD.open("/pokerface.mp3");

I am thinking that if I rename my mp3's to 4 digit names starting from 1000.mp3 then I should somehow be able to contruct a string with random(1000, MaxNumFiles) ".mp3" to pass to the SD.open command. Will try & see what happenes.

the other is for volume control. It runs from 0.0 - 1.0, min - max.

44: volume.setVolume(1.0);

I will try to wire up a potentiometer which will be read through analogInput & then map the result 0 - 1. I'm not sure if map works with non integers, so in that case I guess I could map(0, 10) then divide by 10.

13

The "I need more coffee" update

So of course as with all these things, I said I would walk away for a few days...... turns out that lasted less than a few hours!! However there has been (what I consider personally) huge progress on the audio side of things which pleases me :smiley: I will go into more details shortly but first I will leave a video of my current test.

This video show's monitoring the volume output of each channel to control the brightness of 3 leds:
Blue: The total output volume of the stereo channels. I don't think I will need this, but I might as well implement it as its only 1 extra line of code, so it is there should I need to reference it.
Green: The left channel volume. This is where I will play all of the "vocals" or any other sound that I want to use for synchronizing mouth movement to like squawks etc.
Red: The right channel volume. This channel will be used for playing background music while Fredd13 is singing, or for any other sounds which do not require mouth movement. This channel will also be used for triggering random body movement during songs.

The YouTube video test

The four M's
I have noticed that I am almost working on 3 separate projects simultaneously which will be combined into a single final output as I do not have the knowledge/experience to do this all as one huge project. Each part currently has its own MCU & code for testing purposes, at the end it will all be on a single MCU running the combined code. So the four projects are as follows;

Mood / Mind: This is kind of the brain of the parrot. It is where I am testing sending/receiving commands from the remote via ESP-NOW. It will be handling all the inputs/outputs/triggers responsible for the overall control. While the parrot is in "idle" mode where there is no sound triggers coming in, it will generate commands for random movements / sounds now & then to give the impression that the parrot is alive & doing its own thing. It will also be responsible for handling files on the SD card so that the required folder/file names can be passed to the audio section for playing.

Mechanical: I guess the easiest way to describe this would be handling the input/output of the "physical" parts of the parrot, so interfacing with the motors & switches built into the parrot itself, controlling leds etc.

Music: This is actually all of the audio processing function (but audio does not start with an M!). This will be responsible for receiving filenames & playing them while outputting sound to the speaker as well as volume levels used for movement syncing.

Merge: As you can probably guess by the name, this will be the final stage of the project where I take the previous three sections & combine them together to achieve the overall functioning of the animatronic!

The actual update

Now all of that is out of the way & should hopefully make things a littler clearer in future updates, I will discuss what I have learned / achieved today.
As per previous post I started the day with some code that allowed me to read an mp3 from the SD card & output it to the speaker. So my first goal was to get the volume control working which turned out to be far easier than expected. Wired up a 1k pot to 3.3v, Gnd, P34, gives me a range of 0-4095, which is then mapped to the 0.0 - 1.0 needed using the following code which updates every 250 millis, I can easily drop that to once per second as I do not plan to be adjusting volume that often anyways, mostly just "max volume because I am outside with it" or "lower volume because I am at home testing".

if (millis() >= (volMillis + 250)){
  potValue = float(map((analogRead(potPin)), 0, 4095, 0, 100))/100;
  volume.setVolume(potValue);
  Serial.println(potValue);
  volMillis = millis();
}

However the real headache came when trying to read the output levels of the mp3's. First was a lot of headaches & messing about with the volumeOutput class of the library just to get it running. This probably took me about 4 hours because I do not have the understanding of code well enough yet to figure out where I was going wrong & also what I believe is an error caused by an outdated example. Eventually I got it running, however all 3 channels (mono, left, right) were reading zero, turned out it was not receiving any data to process & finally I managed to make sense of the "add" command which would add send data output to both the I2s & volumeOutput streams.

Great success!!!

Except now it was outputting unusable numbers. In the test video you will notice there is about 10 seconds of each mono, Left, Right, but the output numbers were all roughly the same on all 3 channels no matter which was playing...... all apart from some slight dips during the silent part.
After a couple of hours looking about for information which I could not find...... I was ready to give up & just ask the library author how to do it, but then I stumbled upon the different codecs & decided to just give *.wav a go instead. PERFECT!!! Fired up the Arduino IDE so I could use the serial plotter & I was getting level readings I could use! At the same time I used the potentiometer to adjust the output volume to a lower level & was still getting the same level readings on the plotter weather it was max / min on the pot.
After this it was a simple map function to take the output levels & adjust the 3 WS2812 via FastLED Now that is all working I will be able to playing about with tweaking a "threshold" value which I will pass to the motor controls. The following mapping is a bit messy as I just wanted to get something together quickly to test & this is what worked, obviously when complete the outputs will be held in their own variables etc for easy access.

//    Serial.print("Volume: ");
    Serial.print(volLev.volume());
    Serial.print(",");
//    Serial.print(" left: ");
    Serial.print(volLev.volume(0));
    Serial.print(",");
//    Serial.print(" right: ");
    Serial.println(volLev.volume(1));

leds[1].setRGB(0, 0, (map(volLev.volume(), 0, 20000, 0, 255)));
leds[2].setRGB(0, (map(volLev.volume(0), 0, 20000, 0, 255)), 0);
leds[0].setRGB((map(volLev.volume(1), 0, 20000, 0, 255)), 0, 0);
FastLED.show();

Next Steps (also help please)

Help please first.... The 3W MAX98357 output is just not high enough. I have followed instructions online which say to put a 100k resister between gain / ground to boost to maximum volume. I have also checked the sound files & I can not boost them any further without clipping the sound. The previous 5W mp3 module I was using had a perfect amount of volume but I am having issues finding a suitable I2S module to do the same. This is where I would like some advice if possible, I am thinking that I may need to try & find an I2S module which can output line level volume which is then passed to a separate amplifier. I am a little bit at a loss as to what modules to consider, especially as I have a physical "as small as possible to fit inside the body" limitation. I will continue looking but any bright ideas would be appreciated thanks :slight_smile:

The next steps for the audio system are as follows:
1: I have a collection so far of about 250 sound files which I thought was ready to use. I now know I will need to go through & convert them all to Wav's, easy enough to do but just time consuming.... I will probably get a handful of "test" files converted & leave the rest until I have finished the audio train so I know if I need to make any other tweaks / changes before doing them all.
2: Test a load of files to try & work out what threshold levels count as triggers for movement/syncing etc. Again this is going to have to be a balance between hardware output & file outputs so will give them their own variables to easily tweak changes without going through the whole code.
3: Filesystem..... This is the part I do not have a clue what I am doing, but I do have some idea's which I will start testing out next..... I have 16 buttons on the remote, I have 16 folders on the SD-card, I have variable amounts of sounds in each folder. So I need to write a few lines of code to receive button number, open corresponding folder, select random file, play. My guess is that I will rename the files as 4 digit numbers so I can construct a string along the lines of "/folder_num/ random(min, max files).wav" to pass to the audio player or something like that, either way getting this part working tonight is my goal!

Thanks again & wish me luck :smiley:

Todays code:


#include <SPI.h>
#include <SD.h>
#include "AudioTools.h"
#include "AudioCodecs/CodecWAV.h"
#include <FastLED.h>

#define LED_PIN     27
#define NUM_LEDS    3
#define BRIGHTNESS  255
#define LED_TYPE    WS2812
#define COLOR_ORDER RGB
CRGB leds[NUM_LEDS];
#define UPDATES_PER_SECOND 100
int glow = 255;

const int potPin = 34;
const int chipSelect=5;

float potValue = 0;
int volMillis = 0;


I2SStream i2s; // final i2sput of decoded stream
AudioInfo info(44100, 2, 16);
VolumeStream volume(i2s);
VolumeOutput volLev;
EncodedAudioStream audOut(&volume, new 	WAVDecoder()); // Decoding stream
StreamCopy copier; 
MultiOutput multiOut;
File audioFile;

void setup(){
  Serial.begin(115200);
  //AudioLogger::instance().begin(Serial, AudioLogger::Info);  

  // setup file
  SD.begin(chipSelect);
  audioFile = SD.open("/channeltest.wav");


        // setup multi output
  multiOut.add(volLev);
  multiOut.add(audOut);

  // setup i2s
  auto config = i2s.defaultConfig(TX_MODE);
  i2s.begin(config);

  // setup I2S based on sampling rate provided by decoder
  audOut.setNotifyAudioChange(i2s);
  audOut.begin();

    // set initial volume
    auto Config = volume.defaultConfig();

  volume.begin(config); // we need to provide the bits_per_sample and channels
  volume.setVolume(0.0);
  
  // Volume output begin //
    volLev.begin(info);

  //attach WS812 LED
  FastLED.addLeds<WS2812, LED_PIN, GRB>(leds, NUM_LEDS);
  fill_solid(leds, NUM_LEDS, CRGB::Red);
  FastLED.setBrightness(255);
  FastLED.show();


  // begin copy
  copier.begin(multiOut, audioFile);

}

void loop(){
  if (!copier.copy()) {
    stop();
  }


if (millis() >= (volMillis + 250)){
  potValue = float(map((analogRead(potPin)), 0, 4095, 0, 100))/100;
  volume.setVolume(potValue);
  Serial.println(potValue);
  volMillis = millis();
}

//    Serial.print("Volume: ");
    Serial.print(volLev.volume());
    Serial.print(",");
//    Serial.print(" left: ");
    Serial.print(volLev.volume(0));
    Serial.print(",");
//    Serial.print(" right: ");
    Serial.println(volLev.volume(1));

leds[1].setRGB(0, 0, (map(volLev.volume(), 0, 20000, 0, 255)));
leds[2].setRGB(0, (map(volLev.volume(0), 0, 20000, 0, 255)), 0);
leds[0].setRGB((map(volLev.volume(1), 0, 20000, 0, 255)), 0, 0);
FastLED.show();
}
14

Did you get sound working? I just stumbled onto a integer-only MP3 decoder library, but did not give it a try yet.

15

Very late seeing the thread and admit I haven’t studied it thoroughly. But given your several references to the difficulty of syncing sound to movement, I see no mention of the DFR MP3 Mini Player module? Did you consider that? I’d imagine it would not be too difficult to trigger specific tracks at the start of associated movements.

16

I will try to keep this response short(ish) as I have not really looked at it any more since previous post. Thank you both for the replies :slight_smile:

Doh! To be honest I kid of forgot those modules even existed! So no I have not tried them yet. Apparently I have already spent to much this week so will have to wait until next Thursday (30th) to try any new parts, so if all else fails I think that will be my next option.
Just one note that its kind of the other way around, Its not "trigger movement -----> play sound" but rather "play sound -----> read levels -----> sync movement". That way I can just create the sound files & not have to program animations for each file.

In a word "yes" I now have the audio working as needed........ however:

1: I still need to do work on the "control" side of it like work out how to start/stop sound playing, select file to play, work out when the file ends etc. I imagine that compared to the rest of it this should be relatively easy..... I just have not looked at it yet. Will probably be tonight's goal though.

2: Sound volume/quality. I'm pretty sure the issue is the MAX98357 DAC/Amp. I am using the same speaker as I had connected to the BT201 mp3 module, That had a good volume & quality was good enough for what I need. However using that speaker on the MAX98357 gives poor results...... The volume is low & when it is at maximum volume I am getting bad distortion. I have looked into the bluetooth speaker I use at festivals which gives plenty of volume at good quality as a comparison & that turns out to be a 3w 45mm driver combination. Having a look online I have seen a few people complain about the quality of sound from the MAX98357 but have had better luck with a CS4344 & external amplifier.

The four M's:

Mind: Has not changed since previous post.

Mechanical: The parrot is currently in parts waiting for me to solder some new wires in place. I have also been removing parts I will not use to cut down on weight & add valuable internal space. As previously mentioned the internal mechanism of the head/eye movement means I can not easily convert it to servo motors instead, but I have cleared enough space I may be able to add a servo to add crest style feathers such as in the following video: Headphone warning for loud bird sounds which is an afterthought, but would be easy enough to implement if the servo fits.

Music: As stated, this is moving along slowly but in a positive direction. Mainly need to sort out the "control" code & the quality/volume issues.

Merge: Still a long way off

Although not strictly to do with programming, I thought I would end this post by sharing some "inspiration" of where I want the aesthetics to go with this project.

These 3 videos as far as I can tell all use the original built in controller / electronics, but the aesthetic is exactly what I am after & made me order this toy in the first place.

Locked in A Room

Anywayz Fotografie

Magesteve

Todays final thoughts? Apparently even when I say I am going to give a short response...... I always fail at that too!!!

17

@ controls: you could preprocess the sound files and extract features (level, beat, ...) to a text file that you could use to derive servo motions. In my system I have implemented a compiler that translates motion/gesture commands to movement blocks at loadtime. These blocks (non-overlapping, continous) then that drive the servos in realtime. Trigger events start different precompiled blocks. Reacting to realtime sound is a different thing though, as you have a processing delay.

@ BT-Speakers: did you get them working with ESP32?

@ mp3 modues: these work quite good, but there is a lad that you will have to take care of.

18

Thanks again @zwieblum

@ mp3 modules
@ BT-Speakers

Both of there were only used as "dumb" modules....... I never got as far as "speaking" to them with the ESP32. I was putting the SD-card into the module & using the modules buttons to play the files. The only connection to ESP32 was audio out line feeding the analoginput of the ESP. This voltage level was then what triggered the movement. So the ESP32 was only doing the following:

Threshold = (approximately 10% increase in signal level)

AnalogRead(audioinput)
if (audioInput >= Threshold)
digitalWrite(mouthMotor, HIGH)
else digitalWrite(mouthMotor, Low) // Mouth is spring loaded, if motor stops then mouth springs closed.

Same for body motor, if HIGH motor rotates a cam which makes body move.

This was working fine for tests, Left audio channel contained only the vocals, triggered digital HIGH / LOW for mouth motor.
Right audio channel contained only music, HIGH / LOW body motor.

This all worked perfectly on the tests other than a lot of noise on the analog input, even after passing through a voltage divider. This was also affected by the master volume passed to itm if I turned down the volume on my mp3 module I also lost voltage to the analog input so thresholds were not being crossed.

With that in mind, I switched to the I2S idea, now I am able to digitally read the amplitude of each channel before passing it to the volume control for output. So no analog noise or signal changes when volume control is used.

@ Controls: Honestly that sounds like it might be a little overkill for what I am trying to achieve. I can completely understand how that would have so many benefits on a more advanced animatronic which needs to move servos through sequences of certain positions etc & in the future should I build a fancy animatronic I will definitely look into this way of doing things. But as with this one I am only controlling HIGH / LOW of 2 motors, no fancy audio processing / beat detection etc, only monitoring volume of LEFT / Right channel, which is working to an acceptable level in the channel LED test video, tonight I will try & put the parrot back together again & instead of passing HIGH / LOW to the LED's, I will pass it to the motors instead to see how well it works.

Thanks again :slight_smile:

19

Hm ... the easy way to detect audio levels is a lowpass filter + diode, but when you have i2s working it's ok :slight_smile:

20

Thanks, understood. At the risk of grossly over-simplifying, couldn’t that be achieved with a brief delay between sound and animation? Play track T7.mp3, non-blocking delay, call movement M7?