Unable to get ArduinoSound library to work with Nano 33 BLE boards

For the MP34DT05 PDM microphone we have an inbuilt example sketch - PDMSerialPlotter for the Nano 33 BLE boards (which works fine). Here is the reference to the PDM library.

There the reference quotes -

The library takes care of the audio that will be accessible also through the ArduinoSound Library.

However that isn't quite clear. Does this mean that we can use the PDM class/library with the ArduinoSound classes like FFTAnalyzer class. Example:

if (!fftAnalyzer.input(AudioInI2S))

Here the AudioInI2S class is used by FFTAnalyzer class.

Cause I tried that (replacing AudioInI2S with PDM) and it failed i.e didn't compile for the nano board.

I believe this is an architectural issue as when I change the board to SAMD21 based boards, the issue comes with PDM Library and when the boards are kept as the new Nano 33 BLE boards then the issue comes with ArduinoSound library.

EDIT: Moreover with the Board selected as Arduino Nano 33 BLE, the ArduinoSound library is listed under INCOMPATIBLE.

What I wanted to do was to run FFT on the PDM input of the microphone to determine what frequencies are present.

Any hint would be much appreciated.

Does anyone know where I can get the source files of the PDM Library ? I tried looking in my installation but there is no file by the name of PDM.h. Neither did I find it anywhere on the internet. How is it getting imported ?

I need to do the exact same thing. To no avail I have tried mulitiple libraries and edits. The new BLE sense board is a bit too light on examples for the mic. Anyone have any knowledge as to the structure of the PDM data and maybe a method to do FFT "manually"?

I think I got something close to working. Note I put some limits on it because of the range I was working on. You can just change the lines in the loop for the “println” of “oldx”

/*

Example of use of the FFT libray
Copyright (C) 2014 Enrique Condes

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.

*/

/*
In this example, the Arduino simulates the sampling of a sinusoidal 1000 Hz
signal with an amplitude of 100, sampled at 5000 Hz. Samples are stored
inside the vReal array. The samples are windowed according to Hamming
function. The FFT is computed using the windowed samples. Then the magnitudes
of each of the frequencies that compose the signal are calculated. Finally,
the frequency with the highest peak is obtained, being that the main frequency
present in the signal. This frequency is printed, along with the magnitude of
the peak.
*/

#include “arduinoFFT.h”

// start edit
#include <PDM.h>

// buffer to read samples into, each sample is 16-bits
short sampleBuffer[256];
short oldx;

// number of samples read
volatile int samplesRead;
//stop edit

arduinoFFT FFT = arduinoFFT(); /* Create FFT object /
/

These values can be changed in order to evaluate the functions
/
const uint16_t samples = 256; //This value MUST ALWAYS be a power of 2
const double signalFrequency = 1000;
const double samplingFrequency = 16000;
const uint8_t amplitude = 100;
/

These are the input and output vectors
Input vectors receive computed results from FFT
*/
double vReal[samples];
double vImag[samples];

#define SCL_INDEX 0x00
#define SCL_TIME 0x01
#define SCL_FREQUENCY 0x02
#define SCL_PLOT 0x03

void setup()
{
Serial.begin(115200);
Serial.println(“Ready”);

// start edit
PDM.onReceive(onPDMdata);

// optionally set the gain, defaults to 20
// PDM.setGain(30);

// initialize PDM with:
// - one channel (mono mode)
// - a 16 kHz sample rate(1600hz edit)
if (!PDM.begin(1, 16000)) {
Serial.println(“Failed to start PDM!”);
while (1);
}
// end edit
delay(9000);
}

void loop()
{
/* Build raw data /
double cycles = (((samples-1) * signalFrequency) / samplingFrequency); //Number of signal cycles that the sampling will read
for (uint16_t i = 0; i < samples; i++)
{
vReal = int8_t((amplitude * (sin((i * (twoPi * cycles)) / samples))) / 2.0);/ Build data with positive and negative values*/

//vReal = uint8_t((amplitude * (sin((i * (twoPi * cycles)) / samples) + 1.0)) / 2.0);/* Build data displaced on the Y axis to include only positive values*/
_ vImag = 0.0; //Imaginary part must be zeroed in case of looping to avoid wrong calculations and overflows
* }
/ Print the results of the simulated sampling according to time /
// Serial.println(“Data:”);_
// PrintVector(vReal, samples, SCL_TIME);
FFT.Windowing(vReal, samples, FFT_WIN_TYP_HAMMING, FFT_FORWARD); / Weigh data /
_// Serial.println(“Weighed data:”);_
// PrintVector(vReal, samples, SCL_TIME);
FFT.Compute(vReal, vImag, samples, FFT_FORWARD); / Compute FFT /
_// Serial.println(“Computed Real values:”);_
// PrintVector(vReal, samples, SCL_INDEX);
_// Serial.println(“Computed Imaginary values:”);_
// PrintVector(vImag, samples, SCL_INDEX);
_ FFT.ComplexToMagnitude(vReal, vImag, samples); / Compute magnitudes /
// Serial.println(“Computed magnitudes:”);_
// PrintVector(vReal, (samples >> 1), SCL_FREQUENCY);
_ double x;
double v;
FFT.MajorPeak(vReal, samples, samplingFrequency, &x, &v);
if (v>5000){
if(x<2000)
oldx=x;
}
Serial.println(oldx);*_

//Serial.print(", ");
//Serial.println(v);
_ //while(1); /* Run Once /
//delay(1000); / Repeat after delay /
}
void PrintVector(double *vData, uint16_t bufferSize, uint8_t scaleType)
{_
for (uint16_t i = 0; i < bufferSize; i++)
_ {
double abscissa;
/ Print abscissa value /
switch (scaleType)
{_
case SCL_INDEX:
_ abscissa = (i * 1.0);
break;_
case SCL_TIME:
_ abscissa = ((i * 1.0) / samplingFrequency);
break;_
case SCL_FREQUENCY:
_ abscissa = ((i * 1.0 * samplingFrequency) / samples);
break;
}
Serial.print(abscissa, 6);_
if(scaleType==SCL_FREQUENCY)
_ Serial.print(“Hz”);
Serial.print(" ");
Serial.println(vData, 4);
}
Serial.println();
}
//start edit*

void onPDMdata() {
* // query the number of bytes available*
* int bytesAvailable = PDM.available();
// read into the sample buffer*

* PDM.read(sampleBuffer, bytesAvailable);
for (int i = 0; i < samplesRead; i++) {*_

vReal_=sampleBuffer*;
}
// 16-bit, 2 bytes per sample*

* samplesRead = bytesAvailable / 2;
}
//end edit*_

@johnandrewjustice - you code had a few typos but I managed to get it to compile eventually.

If all I want is a loudness meter (I don't care about frequencies, just the global sound level), then all I need is the computed magnitudes, right? The vReal vector.

Starting with the vReal vector, what would be a good measure of overall "loudness"? Raise all elements to the power of two, compute the sum, extract square root? Or just a simple sum of all elements?

The typos were probably caused by the forum software as the code was not inside code brackets </>
That is why we always ask posters to use them.

EG.

/*
  GPS Location

  This sketch uses the GPS to determine the location of the board
  and prints it to the Serial monitor.

  Circuit:
   - MKR board
   - MKR GPS attached via I2C cable

  This example code is in the public domain.
*/

#include <Arduino_MKRGPS.h>

void setup() {
  // initialize serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  // If you are using the MKR GPS as shield, change the next line to pass
  // the GPS_MODE_SHIELD parameter to the GPS.begin(...)
  if (!GPS.begin()) {
    Serial.println("Failed to initialize GPS!");
    while (1);
  }
}

void loop() {
  // check if there is new GPS data available
  if (GPS.available()) {
    // read GPS values
    float latitude   = GPS.latitude();
    float longitude  = GPS.longitude();
    float altitude   = GPS.altitude();
    float speed      = GPS.speed();
    int   satellites = GPS.satellites();

    // print GPS values
    Serial.print("Location: ");
    Serial.print(latitude, 7);
    Serial.print(", ");
    Serial.println(longitude, 7);

    Serial.print("Altitude: ");
    Serial.print(altitude);
    Serial.println("m");

    Serial.print("Ground speed: ");
    Serial.print(speed);
    Serial.println(" km/h");

    Serial.print("Number of satellites: ");
    Serial.println(satellites);

    Serial.println();
  }
}

Hello @Florin_Andrei can you please give us the right code

@Mustapha_EL_YOUSSOUFY

This is the project I’m working on:

The current version performs FFT on the signal from the sound sensor. Seems accurate enough to me. Code:

/*
 * This code is supposed to run on an Arduino Nano 33 BLE Sense.
 * It makes full use of all sensors on that device.
 * 
 * It reads atmospheric parameters such as:
 * - temperature
 * - pressure
 * - humidity
 * 
 * It reads other environmental parameters such as light intensity.
 * 
 * It reads ambient sound via the onboard microphone. It performs FFT
 * (Fast Fourier Transform) on the sound samples. From the spectrum generated by FFT
 * some frequencies are discarded (those that appear to be outliers).
 * The rest are transformed via a logarithmic function that mimics the human ear
 * and then are summmed up. The sum is taken to be a measure of ambient noise.
 * 
 * Finally, accelerometer and gyroscope data are recorded, along with magnetic field data.
 * 
 * All these measurements are taken several times per second. Individual samples are
 * averaged over one second, then are sent via USB/serial out.
 * 
 * That's all this code does. Take all measurements from all sensors
 * as fast as possible, and send those out (one line = one batch of measurements).
 * 
 * It's up to the receiver side to parse, interpret, and store that data.
 * 
 * Care was taken to prevent system freeze
 * by allowing brief pauses between sensor reads.
 * 
 * Probably not all sensor reads need to be braketed by delay().
 * But it's time-consuming to figure out which need delay().
 * It's probably easier to intersperse delay() among all sensor reads
 * and just tweak the value of the delay() parameter.
 * A minimmum of one week of uninterrupted work is considered the gold standard
 * for stability.
 * 
 * TBD: In the future some kind of watchdog needs to be implemented
 * since not all lock-up situations can be avoided with code shims such as delay().
 * 
 * A very solid code that doesn't seem to need a watchdog would be
 * a solid base for everything else. A watchdog would then be more like insurance.
 */

// HTS221 sensor library
// https://www.arduino.cc/en/Reference/ArduinoHTS221
// https://github.com/arduino-libraries/Arduino_HTS221
#include <Arduino_HTS221.h>
// LPS22HB sensor library
// https://www.arduino.cc/en/Reference/ArduinoLPS22HB
// https://github.com/arduino-libraries/Arduino_LPS22HB
#include <Arduino_LPS22HB.h>
// LSM9DS1 sensor library
// https://www.arduino.cc/en/Reference/ArduinoLSM9DS1
// https://github.com/arduino-libraries/Arduino_LSM9DS1
#include <Arduino_LSM9DS1.h>
// APDS9960 sensor library
// https://www.arduino.cc/en/Reference/ArduinoAPDS9960
// https://github.com/arduino-libraries/Arduino_APDS9960
#include <Arduino_APDS9960.h>
// MP34DT05 sensor library
// https://www.arduino.cc/en/Reference/PDM
// https://github.com/arduino/ArduinoCore-nRF528x-mbedos/tree/master/libraries/PDM
#include <PDM.h>
// TBD
#include <arduinoFFT.h>

arduinoFFT FFT = arduinoFFT();

// store readings from sensors
float temperature, humidity, pressure;
float acc_x, acc_y, acc_z;
float gyro_x, gyro_y, gyro_z;
float magnet_x, magnet_y, magnet_z;

// line output to serial
char linebuf_all[200];

// store readings from light sensor
int r, g, b, w;

// define FFT parameters
#define SAMPLES 256
#define SAMPLING_FREQUENCY 16000
// buffer to read samples into, each sample is 16-bits
short wform[SAMPLES];
// FFT real and imaginary vectors
double vReal[SAMPLES];
double vImag[SAMPLES];

// number of samples read
volatile int samplesRead;

// constrain the APDS readiness loop
short apds_loop;
#define APDS_MAX 50

// final result from FFT
double ftsum = 0.0;

// short pause between sensor reads
short srelax = 40;

int ledState = LOW;

void setup() {
  Serial.begin(115200);
  delay(100);

  // temperature and humidity
  HTS.begin();
  delay(100);

  // pressure
  BARO.begin();
  delay(100);
  // The baro sensor reads wrong first time after init
  // so let's do a throw-away read here.
  pressure = BARO.readPressure(MILLIBAR);
  delay(100);

  // acceleration, gyroscope, magnetic field
  IMU.begin();
  delay(100);

  // light
  APDS.begin();
  delay(100);

  // sound
  PDM.onReceive(onPDMdata);
  delay(100);
  PDM.begin(1, SAMPLING_FREQUENCY);
  delay(100);

  pinMode(LED_BUILTIN, OUTPUT);

  // Let's allow things to settle down.
  delay(100);
}

void loop() {

  apds_loop = 0;
  // always check if sensor is available before reading from it
  while (! APDS.colorAvailable()) {
    // always wait a bit after APDS.colorAvailable()
    delay(srelax);
    // don't get stuck
    if (++apds_loop > APDS_MAX) {
      break;
    }
  }
  if (apds_loop <= APDS_MAX) {
    APDS.readColor(r, g, b, w);
    delay(srelax);
  } else {
    // failed to read, move on
    r = 0;
    g = 0;
    b = 0;
    w = 0;
  }

  temperature = HTS.readTemperature();
  delay(srelax);
  humidity = HTS.readHumidity();
  delay(srelax);
  pressure = BARO.readPressure(MILLIBAR);
  delay(srelax);

  IMU.readAcceleration(acc_x, acc_y, acc_z);
  delay(srelax);
  IMU.readGyroscope(gyro_x, gyro_y, gyro_z);
  delay(srelax);
  IMU.readMagneticField(magnet_x, magnet_y, magnet_z);
  delay(srelax);

  // wait for sound samples to be read
  if (samplesRead) {
    delay(srelax);
    for (int i = 0; i < SAMPLES; i++) {
      // load the waveform into the FFT real vector
      vReal[i] = double(wform[i]);
      // FFT imaginary vector is zero
      vImag[i] = 0.0;
    }

    // compute the spectrum
    // at the end of the sequence, vReal will contain the spectrum
    FFT.Windowing(vReal, SAMPLES, FFT_WIN_TYP_HAMMING, FFT_FORWARD);
    FFT.Compute(vReal, vImag, SAMPLES, FFT_FORWARD);
    FFT.ComplexToMagnitude(vReal, vImag, SAMPLES);

    // calculate the sum of all spectral components
    // with log10() to adjust for perceptual scale
    ftsum = 0.0;
    // don't start i at 0, low frequencies are too noisy
    // stop at sR / 2 since the spectrum is repeated symmetrically after that
    // (that's how FFT works)
    for (int i = 8; i < samplesRead / 2; i++) {
      ftsum += log10(vReal[i]);
    }

    // clear the samples read count
    samplesRead = 0;
  }

  // prepare the line output with all data
  sprintf(linebuf_all,
    "a,%.2f,%.1f,%.2f,g,%.2f,%.2f,%.2f,r,%.2f,%.2f,%.2f,m,%.1f,%.1f,%.1f,l,%u,%u,%u,%u,n,%u",
    temperature, humidity, pressure,
    acc_x, acc_y, acc_z,
    gyro_x, gyro_y, gyro_z,
    magnet_x, magnet_y, magnet_z,
    r, g, b, w,
    int(ftsum));

  // send data out
  Serial.println(linebuf_all);

  // blink the LED every cycle
  // (heartbeat indicator)
  ledState = ledState ? LOW: HIGH;
  digitalWrite(LED_BUILTIN,  ledState);

  delay(srelax);
}

void onPDMdata() {
  // query the number of bytes available
  int bytesAvailable = PDM.available();

  // read into the sample buffer
  PDM.read(wform, bytesAvailable);

  // 16-bit, 2 bytes per sample
  samplesRead = bytesAvailable / 2;
}

@Florin_Andrei

Thank you for the code, it works perfectly, but i still don't understand how to use the arduinofft library.

The library plots in the serial plotter the sum of the fft, but don't give the gain in db and the the frequencies.

The aim of my project is to detect the human breathing frequency using a microphone and for that, i would like to proceed as folow: Record a sample, do an fft on it and print it with serial plotter to have the useful data (a diagram with db(hz))

Do you have a better idea ?

Hey!

Can you use similar code to stream in real time? I am trying to retrofit my doorbell. (as I have a speaker system but no mic)

Starting with the vReal vector, what would be a good measure of overall “loudness”? Raise all elements to the power of two, compute the sum, extract square root? Or just a simple sum of all elements?

For loudness what you need to do is to calculate the RMS value first.

You can do that out of the vReal vector X or directly out of the samples, x, saving you the time consuming FFT.

If you do the FFT, the advantage is that filters can be applied by multiplying the X values with their appropriate weighting scale value. The most well known is the A scale leading to dbA.

If you want to get the dB values, it is important to know that the microphone signal is proportional to the sound pressure P.

This leads to

Lp = 10 log10 (PRMS2) )+ constant1

Lp = 10 log10 (sum ( x2 ) + constant2 = 10 log10 (sum ( X2 ) )+ constant3

The value of the constant can be found by comparing the produced value with a calibrated dB meter. If you don’t have one, there are lots of smartphone apps available with fair results.

Are you still here? :confused:

I’m not hindered by any knowledge of what the library does, but if there’s a DC offset in the produced sample signal it must be corrected for. In other words the microphone signal of an absolute silence should equal 0. If not an offset value must be subtracted before any calculation is done.

Simple example for loudness… works on my Nano 33 BLE Sense

/* For MP34DT05 microphone */
#include <PDM.h>

#include <arm_math.h>

#define MICROPHONE_BUFFER_SIZE_IN_WORDS (256U)
#define MICROPHONE_BUFFER_SIZE_IN_BYTES (MICROPHONE_BUFFER_SIZE_IN_WORDS * sizeof(int16_t))

/* MP34DT05 Microphone data buffer with a bit depth of 16. Also a variable for the RMS value */
int16_t microphoneBuffer[MICROPHONE_BUFFER_SIZE_IN_WORDS];
int16_t microphoneRMSValue;

/* Used as a simple flag to know when microphone buffer is full and RMS value
 * can be computed. */
bool microphoneBufferReadyFlag;

void setup()
{
  Serial.begin(230400, SERIAL_8O1);
  while(!Serial);

  /* Initialise microphone buffer ready flag */
  microphoneBufferReadyFlag = false;

  /* PDM setup for MP34DT05 microphone */
  /* configure the data receive callback to transfer data to local buffer */
  PDM.onReceive(Microphone_availablePDMDataCallback);

}

void loop()
{


 ... Other stuff


 /* If the microphone buffer is full, compute the RMS value */
 if(microphoneBufferReadyFlag)
 {
   Microphone_computeRMSValue();
   microphoneBufferReadyFlag = false;
 }
}

void Microphone_availablePDMDataCallback()
{
 // query the number of bytes available
 int bytesAvailable = PDM.available();

 if(bytesAvailable == MICROPHONE_BUFFER_SIZE_IN_BYTES)
 {
   PDM.read(microphoneBuffer, bytesAvailable);
   microphoneBufferReadyFlag = true;
 }
}

void Microphone_computeRMSValue(void)
{
 arm_rms_q15((q15_t*)microphoneBuffer, MICROPHONE_BUFFER_SIZE_IN_WORDS, (q15_t*)&microphoneRMSValue);
}

davzarek:
Can you use similar code to stream in real time?
I am trying to retrofit my doorbell. (as I have a speaker system but no mic)

Currently the BLE standard does not support audio because it does not have enough bandwidth. You could create your own proprietary solution and limit the bandwidth and resolution of your audio until you can send it over BLE. Whether this is good enough for your use case is something you need to find out.

ian_west:
Simple example for loudness… works on my Nano 33 BLE Sense

It doesn’t work on mine.

Even after tweaking I still can’t get it to work, e.g. addition of

if (!PDM.begin(1, 16000)) {
Serial.println(“Failed to start PDM!”);
while (1);
}

in the setup.

Do you have some runnable code?

Do you have some runnable code?

. Short answer yes. Long answer, I was unable to transmit the audio as fast as it was acquired, and therefore the solution I used was to buffer the audio, then transmit it over serial, once the buffer is full.

Here's a really simple sketch to get audio from the Nano 33 BLE Sense, and transmit it back to a connected desktop over serial.

https://github.com/victorromeo/arduino_micro_speech/blob/master/data_in/arduino_pdm/arduino_pdm.ino

If it is of interest to anyone here, there are also examples how to use the audio to train a TF2.0 + Keras SB-CNN neural network, and include it in a variant of the micro_speech project included in the repo also, including my Nano 33 BLE Sense acquired audio samples and results.

Here’s a really simple sketch to get audio from the Nano 33 BLE Sense, and transmit it back to a connected desktop over serial.

It looks like an adaptation of the even simpler example that comes with the installation of the PDM library called PDMSerialPlotter. Which runs fine btw. My serial connection seems to cope with the amount of information, but even in the monitor the image flashes for your eyes.

It would be a lot better to do some processing and send that, like in your example that I did not manage to get working. I find it difficult to merge the two examples, which boils down to the value of samplesRead.

AFAIK the the serial port setting for the Nano BLE is irrelevant, as it communicates directly over the USB, ignoring restricting serial communication settings.

I got your example to compile and run but the returned values remain 0

/* For MP34DT05 microphone */
#include <PDM.h>

#include <arm_math.h>

#define MICROPHONE_BUFFER_SIZE_IN_WORDS (256U)
#define MICROPHONE_BUFFER_SIZE_IN_BYTES (MICROPHONE_BUFFER_SIZE_IN_WORDS * sizeof(int16_t))

/* MP34DT05 Microphone data buffer with a bit depth of 16. Also a variable for the RMS value */
int16_t microphoneBuffer[MICROPHONE_BUFFER_SIZE_IN_WORDS];
int16_t microphoneRMSValue;

/* Used as a simple flag to know when microphone buffer is full and RMS value
 * can be computed. */
bool microphoneBufferReadyFlag = false;

void setup()
{
  Serial.begin(230400, SERIAL_8O1);
  while(!Serial);
  Serial.println("Started"); 

  /* PDM setup for MP34DT05 microphone */
  /* configure the data receive callback to transfer data to local buffer */
  PDM.onReceive(Microphone_availablePDMDataCallback);

  if (!PDM.begin(1, 16000)) {
    Serial.println("Failed to start PDM!");
    while (1);
  }
  Serial.println("PDM OK"); 
}

void loop()
{
 /* If the microphone buffer is full, compute the RMS value */
 if(microphoneBufferReadyFlag)
 { 
   Microphone_computeRMSValue();
   microphoneBufferReadyFlag = false;
   Serial.println(microphoneRMSValue); 
 }
}

void Microphone_availablePDMDataCallback()
{
 // query the number of bytes available
 int bytesAvailable = PDM.available();

 if(bytesAvailable == MICROPHONE_BUFFER_SIZE_IN_BYTES)
 {
   PDM.read(microphoneBuffer, bytesAvailable);
   microphoneBufferReadyFlag = true;
 }
}

void Microphone_computeRMSValue(void)
{
 arm_rms_q15((q15_t*)microphoneBuffer, MICROPHONE_BUFFER_SIZE_IN_WORDS, (q15_t*)microphoneRMSValue);
}

Hi, has anyone come across something like a clap detection, e.g. claculating claps/sounds over a certain amplitude etc.?

I was also looking at: https://www.arduino.cc/en/Tutorial/ArduinoSoundClapDetector

But is do not as other states compute for BLE 33 sense.