I would like to make some simple "robots" where a mouth moves in sync to some speech signal.
So I can play out audio, and I can move a servo. I am also happy making a simple algorithm to sync the audio to servo movement. But, I am unsure how the most efficient way make the servo move and the audio come out in the arduino code.
The only way I can think is to play out e.g. 64 samples of audio, and move the mouth a tiny amount corresponding to that time period, and then go round the loop again. If I do this, I would like some confidence that the main arduino loop time is the same as the audio playback. Otherwise I would need to make a separate loop with a buffer counter that outputs x samples of audio when full, and moves the mouth a little.
But perhaps there is a better way of doing this?
If anyone has any recommendations on doing this, specifically for the Arduino, I would be very happy to hear it.
Thanks
You could do something similar to what was done so many years ago to sync a slide projector to an audio tape recording. Add a short tone burst that way above the normal hearing frequency to the audio on the tape. When the tone was sensed, advance the slide projector one frame.
You could add two toned to your recording. One to open the mouth and a different tone to close the mouth. Or a single tone could alternate open and close.
That seems pretty simple to me.
That will not work, the Audio signal is way too fast compared to mouth movement. What you need is a peak detector driven from the Audio. Then use that to move the mouth.
I did this on a translator a few years back using an LED matrix display but it is the same thing. Basically press a number on a keypad and a pre recorded message comes out it Italian. I used it for answering common questions at the Rome Maker Faire, one year.
The bigger the envelope the audio, the wider the mouth was displayed. I think there are about eight different mouth sizes.
Thanks @Grumpy_Mike this is exactly what I want to do, but I do not intend to move the mouth fully for each block of e.g. 64 samples. I thought I could compute (based on the envelope, or whatever algorithm I use for this) the desired position of the servo at 64 sample intervals, and then (after some quantisation for the servo resolution) move the servo that tiny bit. Then the main arduino loop is always 64 samples. Do you think this is the right way to go?
Without using the envelope the signal would contain both positive and negative deviations from the bias level of the signal. So it would be no good.
Calculating the envelope would involve finding the peak value by software, something that would take far too long to keep up with the audio signal. In other words you could not do this on many, if any types of Arduino.
Also remember you have interrupts going off very regularly due to the signal needed to move or keep the servo at a fixed position, so making any form of calculation based on the individual audio samples all but impossible.
Where as a simple envelope follower on the audio does all that in hardware.
You might even have to farm the servo pulse generation off to and external chip anyway to get it to function.
