Audio Visualization via FFT on Due

This is a learning project - doing the steps from scratch is more important than polished performance of the final product.

Objective: Control brightness of three LEDs based on the signal strength in the low/mid/high frequency range of an analog audio signal.

What I can do already:

  • My home-cooked FFT (written in the Arduino IDE) works correctly on discretized sinusoidal functions. It may well be slow compared to more professional products. So far I have only tested it by printing out the FFT array.
  • I can feed byte blocks from an mp3-file via Python; I can visualize the volume of the entirety of the signal. (Tested with a short techno mp3 - yes, the beat indeed drops.)

As a mathematician (as opposed to an engineer), I'd like to ask for some pointers on where to pick up some knowledge regarding the next step: feeding in an audio signal via either audio output (headphone jack) or USB (like I would for running a USB headset).

I am vaguely aware of the following:

  • Analog audio signals are symmetric AC signals and need to be biased around 1.5-ish volts and kept 0 to 3-ish Volts peak-to-peak to be used by the analog input pins for the Due. For this, I'd like to deploy a simple solution (we are not talking about high-fidelity signal processing here) that can be done on a breadboard.

I have no knowledge at all of the properties of the USB audio signal that drives a headset, but again, I'm not asking you to teach that to me, just to make my search for tutorials a bit more focused.

Here's my block-reading and processing loop, if you are interested:

const int numberLED = 3;
const int buffersPerPacket = 32;
const int bufferSize = 80;
const int packetSize = buffersPerPacket * bufferSize;

const int analogOutPin[numberLED] = {9, 10, 11};  // Analog output pin that the LED is attached to
unsigned int packetData[packetSize] = {};
unsigned int channelData[numberLED] = {};  // value output to the PWM (analog out)
unsigned char buffer[packetSize] = {}; 

void setup() {
  // initialize serial communications at 9600 bps:
  for(int k=0; k<numberLED; k++){
    pinMode(analogOutPin[k], OUTPUT);
  }
  Serial.setTimeout(1);
  Serial.begin(500000);
}

void processPacket(unsigned int packetData[packetSize], unsigned int channelData[numberLED])
{
// omitted, works as intended
}

void loop() {
  // Do nothing while there is not enough data
  while (Serial.available() < bufferSize){;}

  // Read sound packet into array
  for (int b=0; b<buffersPerPacket; b++){
    Serial.readBytes(buffer, bufferSize);
    for(int k=0; k<bufferSize; k++){
      packetData[b*bufferSize + k] = buffer[k];
    }  
    while (Serial.available() < bufferSize){;}
  }

  // Send packet for processing, receive channel data for three LEDs
  processPacket(packetData, channelData);

  // Set brightness of three LEDs
  for(int k=0; k<numberLED; k++){
    analogWrite(analogOutPin[k], channelData[k]);
  }  
}

And here's the Python loop feeding the blocks from mp3:

#mp3 processing omitted

ardu = serial.Serial('/dev/cu.usbmodem14101', baudrate=500000, timeout=.1)
packetSize = 80
numberOfPackets = 10000

# bsignal contains the sound in the range from 0 to 255. 
#signal contains the blocks, ready to be shipped one-by-one via a struct
signal = [bsignal[i:i+packetSize] for i in range(0, packetSize*numberOfPackets, packetSize)]
time.sleep(1)


start_time = time.monotonic()
for sig in signal:
    ardu.write(struct.pack(str(packetSize)+'c', *sig))
end_time = time.monotonic()
print(timedelta(seconds=end_time - start_time))

ardu.close()

You will quickly gain a load of knowledge by studying the audio trace on an oscilloscope.

This circuit is basically a voltage divider and with equal-value resistors the voltage is divided in half (you can apply 3.3V). The capacitor blocks DC from the audio source and it keeps the audio connection from fouling-up the DC voltage.

It doesn't affect sound quality other adding the DC bias (which would be undesirable with for regular audio equipment).

Audio Input Schematic

A USB audio would be "difficult". You'd need to build a class compliant USB device to work with your existing drivers and then the operating system can normally use only one audio device at a time.

I'm not clear on what you're doing with MP3 but the MP3 has to be decompressed/decoded. (I assume there is a Python library or function for that but I've never used Python.) File compression is similar to encryption and the raw data is garbage/noise.

If you are writing your own FFT (DFT) you may already know the basics, but if not, the Audacity website has a nice little introduction to digital audio.

Perfect! It's aways best to work on the input, processing, and output separately before putting everything together.

Audacity can also generate sine waves at various frequencies for testing.

I've never used the FFT library but from what I understand, on the "regular Arduino", a bit of audio is read-in, then analyzed, and the display is updated. So there are gaps and it's not running continuously but it works OK as a "visualizer" or "effect".

But the Due may be fast-enough to do the processing and display updates in-between sample reads...

And if you don't know this, the higher-frequency bins are usually combined so your display works in octaves, or fractions or multiples of octaves.

1 Like

Thank you! This is very helpful in getting me started! I guess experience will show whether the resulting plain (biased) signal/the resolution of the pin are enough to create meaningful data.

Also, yes - the Python library restores the original data from the compression as signed-int array. I shift it and scale it to get plain old bytes. It is about twice slower than the original audio (runs about a minute instead of 30 seconds), but I can see the "beat" in my LEDs, so I'm calling it "correct."

Thank you also for the warning against tinkering with USB. I do indeed not want to go to the deep end with this. But could I not do something with the Due's second USB port (not the programming port)? Say, Audio-OUT from the computer, AUDIO-IN to the Arduino USB port? Would that not be quicker than shoveling bytes from files back and forth?

Almost certainly not. Normally a USB host is required to interface with any USB device, and in this case, the Arduino would have to be a USB device, pretending to be a speaker or headphone.

If you are currently using the Due ADC to measure an analog signal, for transformation with FFT, keep in mind that any data transported over USB are digital.

When you plug your USB headset into an PC, what sort of device comes up in the Device Manager, under the USB heading?

The headphones look like this, although I am unable to identify them among the devices under /dev.

Screen Shot 2023-10-01 at 19.30.55

But that's okay - I am not wedded to using the USB port for this project.

Definitely a digital audio USB device. The chance of making a PC to Due audio USB connection is close to zero, unless you can find someone to write a driver.

1 Like

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.