FHT Results Not As Expected

Using the Adafruit "Electret Microphone Amplifier - MAX4466 with Adjustable Gain", powered by 5V, output to A0 input on a Nano. Running the following code from OpenMusicLabs...

/*
fht_adc_serial.pde
guest openmusiclabs.com 7.7.14
example sketch for testing the fht library.
it takes in data on ADC0 (Analog0) and processes them
with the fht. the data is sent out over the serial
port at 115.2kb.
*/

#define LOG_OUT 1 // use the log output function
#define FHT_N 128 // set to 128 point fht (gives 64 freq bins)
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))

#include <FHT.h> // include the library

unsigned long StartTime = 0;
unsigned long EndTime = 0;

void setup() {
// set prescaler to 32 for a sample rate of 38.4kHz (BW = 19.2kHz)
   sbi(ADCSRA,ADPS2);
   cbi(ADCSRA,ADPS1);
   sbi(ADCSRA,ADPS0);

/*
Prescale  ADPS2,1,0  Clock MHz)  Sampling rate (KHz)
  2 	  0 0 1 	8 	  615
  4 	  0 1 0 	4 	  307
  8 	  0 1 1 	2 	  153
  16 	  1 0 0 	1 	  76.8
  32 	  1 0 1 	0.5 	  38.4 <
  64 	  1 1 0 	0.25 	  19.2
  128 	  1 1 1 	0.125 	  9.6 (default)
*/
 
  Serial.begin(9600); // use the serial port
  TIMSK0 = 0; // turn off timer0 for lower jitter
  ADCSRA = 0xe5; // set the adc to free running mode
  ADMUX = 0x40; // use adc0
  DIDR0 = 0x01; // turn off the digital input for adc0
} // end Setup

void loop() {
  while(1) { // reduces jitter
    cli();  // UDRE interrupt slows this way down on arduino1.0
    for (int i = 0 ; i < FHT_N ; i++) { // save 256 samples
      while(!(ADCSRA & 0x10)); // wait for adc to be ready
      ADCSRA = 0xf5; // restart adc
      byte m = ADCL; // fetch adc data
      byte j = ADCH;
      int k = (j << 8) | m; // form into an int
      k -= 0x0200; // form into a signed int
      k <<= 6; // form into a 16b signed int
      fht_input[i] = k; // put real data into bins
    }
    fht_window(); // window the data for better frequency response
    fht_reorder(); // reorder the data before doing the fht
    fht_run(); // process the data in the fht
    fht_mag_log(); // take the output of the fht
    sei();

    Serial.print("Start:  ");
    for (byte i = 0 ; i < FHT_N/2 ; i++) {
      Serial.print(fht_log_out[i]); // send out the data
      Serial.print(" ");
    }

     Serial.println();
  }
}

With relative quiet at the mic, getting results like...

Start: 129 169 142 115 130 109 82 117 107 95 83 98 87 76 93 86 94 80 76 88 66 74 85 85 76 82 75 74 74 74 71 75 74 65 68 70 63 68 73 70 66 69 70 67 70 68 67 67 65 63 64 60 65 66 62 65 62 64 64 65 64 63 66 65

...with the numbers bouncing around some, which I'd expect due to noise. However, when I whistle into or tap the microphone, I expect to see at least some of the bin values change significantly, but they do not. With a scope at input A0, there is an audio signal representative of what the mic hears, on a DC offset of 2.5V.

With mic disconnected and input A0 grounded, see mostly "0" for results which seems correct and tells me that the FHT is doing something.

Any idea why I don't see values change with a change in audio input? It's acting as though the mic is dead, but scope shows it isn't.

Here's what I'd recommend for a first step: Test the signal path. In quiet surroundings, print the elements of fht_input. Then, print them again while making noise - a more or less steady tone, like a whistle, might be best. Graph both sets of samples in a spreadsheet program, and compare them. Can you see the input signal on the second set?

It's hard to see any changes in specific frequencies when you have the data spread across two lines and each bin can be 2 or 3 chars wide. Reduce the size of the FHT_N to 32 and change this:

   Serial.print("Start:  ");
    for (byte i = 0 ; i < FHT_N/2 ; i++) {
      Serial.print(fht_log_out[i]); // send out the data
      Serial.print(" ");
    }

to this:

    for (byte i = 0 ; i < FHT_N/2 ; i++) {
      static char stmp[16];
      sprintf(stmp,"%3d ",fht_log_out[i]); // send out the data
      Serial.print(stmp);
    }

When you whistle into the mic, you should see a few of the bins increase in size but be aware that they won't change a huge amount because you are printing the log of the magnitudes.

When you've satisfied yourself that it is responding, change FHT_N back to 128.

Pete

Thanks for the help.

Tried printing fht_input and get values like...

Start: 7050 540 355 229 198 178 149 134 126 109 96 88 79 74 65 61
Start: 644 157 41 4 10 44 3 19 20 5 4 13 13 3 10 1
Start: 4754 308 144 123 99 82 73 51 50 44 42 34 25 28 33 27
Start: 1390 -1844 -790 -387 -294 -297 -266 -227 -202 -195 -172 -160 -143 -128 -126 -115
Start: -1754 529 501 332 308 266 223 190 170 163 157 137 124 112 102 90
:
:

Then with fht_lin_out (for more change in the values than log)...

Start: 3936 904 278 328 219 175 132 111 108 90 86 74 85 72 79 80
Start: 398 382 470 332 83 118 52 62 55 51 43 44 33 42 38 36
Start: 1720 202 153 80 59 38 14 8 14 10 13 11 15 2 12 10
Start: 1856 484 334 215 139 157 106 76 70 55 53 50 50 34 45 40
Start: 1544 206 97 142 2 23 45 29 27 25 27 23 23 28 29 16
:
:

Room is relatively quiet, and a tone from my phone close to the mic makes no difference. Maybe there's an issue with the mic. I have another I can try.

Thanks so much for your help! Appears the microphone noisy giving unpredictable results. Replaced with the Adafruit MAX9814 Mic/Preamp/AGC module and now getting expected results.

With "#define FHT_N 64" and Sample Rate set for 38.4kz (Prescale of 32), getting 32 bins across 19.2kHz or 600Hz/bin. With a 1800Hz tone, the 3rd bin expectedly shows energy (6528). Bins track frequency all the way up to 19kHz. Using "fht_mag_lin()" for now instead of "fht_mag_log()" since it shows more of a change than log.

Start: 22144 247 42 6528 92 83 45 34 38 44 34 4 39 53 82 46 49 38 72 102 57 72 27 68 17 30 46 52 38 34 114 52
Start: 22400 260 193 6528 49 27 106 94 110 20 62 87 61 77 48 48 53 56 97 24 77 89 51 43 38 56 43 34 9 42 54 52
Start: 22912 107 160 6528 95 105 47 98 23 78 54 32 58 90 57 48 65 62 93 18 91 58 27 53 138 11 36 109 58 53 80 113
Start: 22528 62 43 6528 88 70 62 36 63 17 66 81 78 107 113 36 108 100 60 57 55 106 79 38 99 22 64 43 50 64 9 47
Start: 21760 140 153 6528 72 107 88 24 26 25 38 95 62 47 33 53 69 113 37 89 47 14 60 76 36 25 82 72 10 66 33 69

Now to tackle the Goertzel algorithm, as I want to also detect a few tones below 150 Hz. Hopefully both Goertzel and FHT can coexist in the same sketch.

So now that I have the Prescaler set to "101" for a 38.4kHz sample rate, I can't seem to change it. Would like to go to 19.2kHz but Prescaler doesn't appear to be changing.
This code got it set to "101" (sbi, cbi, sbi)for a 38.4kHz sample rate...

 :
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))

// set prescaler to 32 (101) for a sample rate of 38.4kHz (BW = 19.2kHz)
// First clear the bits...
   cbi(ADCSRA,ADPS2);
   cbi(ADCSRA,ADPS1);
   cbi(ADCSRA,ADPS0);
// Then set the desired bits...
// sbi = "Set Bit", cbi = "Clear Bit"
   sbi(ADCSRA,ADPS2);
   cbi(ADCSRA,ADPS1);
   sbi(ADCSRA,ADPS0);

But this code won't set it to "110" (sbi, sbi, cbi) for 19.2kHz...

// set prescaler to 64 (110) for a sample rate of 38.4kHz (BW = 19.2kHz)
// First clear the bits...
   cbi(ADCSRA,ADPS2);
   cbi(ADCSRA,ADPS1);
   cbi(ADCSRA,ADPS0);
// Then set the desired bits...
// sbi = "Set Bit", cbi = "Clear Bit"
   sbi(ADCSRA,ADPS2);
   sbi(ADCSRA,ADPS1);
   cbi(ADCSRA,ADPS0);

Don't see what's wrong. Can tell the Prescaler isn't changing because the tone sent in isn't changing bins.Are there some combinations of Prescaler bits and number of bins that aren't accepted?

Been referring to this table fir the bit values...

Prescale  ADPS2,1,0  Clock(MHz)  S.rate(kHz)  BW(kHz)
  2 	      0 0 1 	8 	  615          307        
  4 	      0 1 0 	4 	  307          153
  8 	      0 1 1 	2 	  153          76.8
  16 	      1 0 0 	1 	  76.8         38.4
  32 	      1 0 1 	0.5 	  38.4         19.2 <<
  64 	      1 1 0 	0.25 	  19.2          9.6
  128 	      1 1 1 	0.125 	   9.6          4.8 (default)

Be careful posting only portions of your code. This time, you've omitted the statement that causes the problem.

The clock select bits ADPS2:0 are in register ADCSRA. Is there anything in your code that might affect their values, after you so painstakingly set them?

You are absolutely right! Farther down in the code are...

:
ADCSRA = 0xe5; // set the adc to free running mode
:
ADCSRA = 0xf5; // restart adc

Which sets ADPS2,1,0 to "5 (101) for a 38.4kHz sampling rate.
I should replace that register setting with a bit mask so can change the ADPS2,1,0 bits in one place.

Do you know if ADPS2,1,0 can be changed on the fly to do sampling at two different rates in the same code?

rickso234:
Do you know if ADPS2,1,0 can be changed on the fly to do sampling at two different rates in the same code?

I don't find any description in the datasheet of how that would work. I've never tried it.

I'd certainly expect that you could do it by stopping the ADC, disabling it with ADEN, changing the prescaler, and restarting the ADC. For changing the conversion rate suddenly, you might have better luck using a timer to trigger conversions, and managing the timer's rate. As an alternative, if the rates you want differ by an integer factor, you can let the ADC free-run at a single rate, and selectively ignore readings that you don't need. For example, converting at the default rate of 9.6 kHz should be the same as converting at 19.2 kHz and ignoring every other sample, exclusive of effects on accuracy due to a prescaled ADC clock speed above 200 kHz.

I'd think that changing the prescaler during a conversion would be a bad thing. The hardware may preclude that by buffering the ADPS2:0 bits until a conversion is complete, but, if so, the datasheet doesn't mention it. Because a free-running ADC is always converting, any change to the prescaler would impact a conversion. I'd think that both the accuracy and the timing of analog data would be suspect, if not downright unreliable.

What do you intend to accomplish with that?

Edit: spelling

What do you intend to accomplish with that?

Was hoping to use both a lower sample rate (9.6kHz) and more bins to get maximum low frequency resolution (37.5Hz bins to 4.8kHz) and higher sample rate for a wider frequency range (300Hz bins to 19.2kHz)... sort of the best of both worlds.

sort of the best of both worlds.

No, this will make things worse.

For FFT, FHT, etc. transforms to work, it is assumed and required that the sample rate is constant and accurately known.

You could compute 2 separate FFTs on separate data at 2 different sample rates, given enough memory and CPU time, and of course good data at both sample rates.

But getting the slower sample rate data from the faster data isn't simple. You can't simply discard (also called "decimate" in DSP lingo) samples, because that results in Nyquist aliasing. You need to first low-pass filter away all the high frequencies, which adds another expensive step, especially if you want the digital filter to have a fairly narrow transition band.

On top of all that, you also need to deal with spectral leakage. From your comments so far, it seems you're imagining each FFT bin giving you unique, independent data. That optimistic scenario is only true if your waveform is perfectly repetitive and synchronous with the FFT interval, which only is true in certain very special cases like perfectly synchronized test waveforms or AC power line waveforms where the sample rate is perfectly phase locked (or resampled) to an exactly synchronized multiple of the waveform.

Analysis of music or ordinary sounds certainly won't fit these ideal conditions. You'll almost certainly have to apply a window to avoid contaminating all your data with terrible spectral leakage. That necessarily means a loss in resolution, meaning you'll need even more FFT bins to achieve your goals.

Maybe this isn't the advice you want to hear, and maybe you won't be able to hear it, but the sad reality is your Nano with 8 bit AVR is woefully underpowered for this sort of real-time signal analysis.

Time and time again, people try things like subsampling to lower rates without filtering and using FFT on arbitrary data without windowing, with foolish optimism that utterly ignores very well established DSP fundamentals. It never ends well.

Don't ignore these important DSP principles, no matter how badly you might wish for that underpowered chip to perform these tasks.

rickso234:
Do you know if ADPS2,1,0 can be changed on the fly to do sampling at two different rates in the same code?

Here is yet another case where you need to be aware of Nyquist sampling theory.

Sampled data can represent frequencies only up to half the sample rate. If your analog signal contains any frequencies higher than this limit, they don't magically go away just because you don't want them. Instead, they will "alias", becoming erroneous data at lower frequencies. The end result is horrible distortion in your sampled data.

So even if you can change the sample rate, if your analog signal has higher frequencies which of course are your motivation for sometimes sampling faster, when you switch to a slower rate those frequencies will utterly corrupt the data you acquire.

To make this sort of thing work, you'd need to build analog filters. Perhaps feed the low-pass filtered signal to a different ADC pin and also switch which input you're reading when you switch the sample rate.

Also consider, with a single on-chip ADC, you'll be effectively "blind" to events happening on the full bandwidth signal during the time you've repurposed the ADC to sampling the low bandwidth signal.

...your Nano with 8 bit AVR is woefully underpowered for this sort of real-time signal analysis.

Yes, beginning to realize that. Forgot about Mr. Nyquist, so multiple sample rates is impractical. Looked at memory usage and that's an issue also. Could switch to a more powerful processor.

Do DFT/Goertzel algorithms and IIR filters have the same issues as FHT/FFT with speed, aliasing and memory usage or are the same laws of physics involved? Looking to identify three discrete frequencies (50, 140, & 225 Hz) and two bands (4000-1250Hz & 1800-9000Hz) of frequencies, which might be more efficiently accomplished with DFT/Goertzel and IIR filters.