My spectrum analyzer is noisy!

I wrote a spectrum analyzer for my ESP32 NodeMCU and it's fast and happy and pretty, but it's not accurate or a pretty as it should be because it is very noisy.

If you play a pure sine tone, you'll get a peak one one bar and then randomness to about half that bar's height on the other bars. Adjusting my noise floor can help, but doesn't solve it.

I figured the code might be of use to someone, particularly if we can make it work properly, so I'll share it here. Please take a look at the FFT and comment on what I might be doing wrong!

Thanks,
Dave

class Analyzer
	{
	  private:
		
		static const int SAMPLES = 512;
		static const int SAMPLING_FREQUENCY = 32000; // Hz * 2, so max frequency measured is half the sampling frequency
		static const int NOISE_CUTOFF = 1000;
		arduinoFFT _FFT;

		unsigned int _sampling_period_us;
		unsigned long _microseconds;
		byte _peak[MAX_BANDS] = { 0 }; 
		double _vReal[SAMPLES];
		double _vImaginary[SAMPLES];
		unsigned long _newTime;
		unsigned long _oldTime;
		int dominant_value;

		uint8_t _inputPin;

		static int * BandCutoffTable(int bandCount)
		{
			if (bandCount == 8)
				return cutOffs8Band;
			if (bandCount == 16)
				return cutOffs16Band;
			if (bandCount == 32)
				return cutOffs32Band;
			Serial.println("Error: Bogus bandCount");
		}

		int BucketFrequency(int iBucket)
		{
			if (iBucket == 0)
				return 0;

			int iOffset = iBucket - 2;
			return iOffset * (SAMPLING_FREQUENCY / 2) / SAMPLES;
		}

	  public:

		Analyzer(uint8_t inputPin) 
		{
			_inputPin = inputPin;
			_sampling_period_us = round(1000000 * (1.0 / SAMPLING_FREQUENCY));
		}

		byte * GetBandPeaks()
		{
			return _peak;
		}

		void SampleAudio(float deltaTime, int bandCount)
		{
			for (int i = 0; i < bandCount; i++)
			{
				_peak[i] = 0;
			}

			float expectedTime = SAMPLES * _sampling_period_us;
			unsigned long startTime = micros();

			for (int i = 0; i < SAMPLES; i++)
			{
				_newTime = micros();
				_vReal[i] = analogRead(_inputPin);
				_vImaginary[i] = 0;
				while ((micros() - _newTime) < _sampling_period_us)
				{
					// BUGBUG busy waiting!
				}
			}

			Serial.printf("Sampling took %d us and was expedcted to take %d", micros() - startTime, (unsigned long)expectedTime);

			int * Bands = BandCutoffTable(bandCount);

			_FFT.Windowing(_vReal, SAMPLES, FFT_WIN_TYP_HAMMING, FFT_FORWARD);
			_FFT.Compute(_vReal, _vImaginary, SAMPLES, FFT_FORWARD);
			_FFT.ComplexToMagnitude(_vReal, _vImaginary, SAMPLES);

			// Magical noise filter from https://www.youtube.com/watch?v=5RmQJtE61zE
			// The Goggles, they do nothing!  So commented out for now.
			// for (int i = 2; i < SAMPLES / 2; i++)
			//	_vReal[i] = sqrt(_vReal[i] * _vReal[i] + _vImaginary[i] * _vImaginary[i]);
			
			double overallPeak = 0.0f;
			for (int i = 2; i < SAMPLES / 2; i++)
				if (_vReal[i] > overallPeak)
					overallPeak = _vReal[i];

			float amplitudeScaling = overallPeak / 256.0f;			// We want all results to scale down to a byte
			if (amplitudeScaling <= 32.0f)					// Some reasonable "max" amplifiication for quiet areas
				amplitudeScaling = 32.0f;

			Serial.printf("Peak: %f, Scaling: %f\n", (float)overallPeak, amplitudeScaling);

			for (int i = 2; i < SAMPLES / 2; i++)
			{
				if (_vReal[i] > NOISE_CUTOFF)
				{
					int freq = BucketFrequency(i);

					int iBand = 0;
					while (iBand < bandCount)
					{
						if (freq < Bands[iBand])
							break;
						iBand++;
					}
					if (iBand > bandCount)
						iBand = bandCount;

					float scaledValue = _vReal[i] * scalars16Band[iBand] / amplitudeScaling;
					byte byteVal = scaledValue > 255 ? 255 : (byte) scaledValue;

					if (byteVal > _peak[iBand])
					{
						_peak[iBand] = byteVal;
					}
				}
			}
		}
	};

Img_7290.jpg

You should probably use a logarithmic scale for your display.
Did you use a windowing function? Do you have an anti-aliasing filter?
Your input is real, so an FHT would be much more efficient than an FFT.

Pieter

Makes sense but no one else seems to apply a log scale, though I can try it!

I do use windowing, but with the defaults, if that matters.

I do not have a good anti-aliasing filter as I've not found one in another spectrum analyzer that works!

davepl:
Makes sense but no one else seems to apply a log scale, though I can try it!

Audio level indicators usually have a logarithmic scale. (E.g. VU meters and decibel meters.)

Can someone suggest a function I can apply to the data to convert it from log to linear?

			for (int i = 0; i < SAMPLES; i++)
			{
				_vReal[i] = func(_vReal[i]);
			}

But what is func? I tried natural log and log2 and log10, both seem to scale WAY too much, so I'm not really sure what I'm looking for!

I do not have a good anti-aliasing filter as I've not found one in another spectrum analyzer that works!

Aliasing leads to what appears to be noise in a spectrum.

To avoid aliasing, you MUST NOT allow signals with frequency higher than (sample frequency)/2 to reach the input.

To test your FFT function, create an ideal data stream with known signals that obey the rules, and offer that as input.

I really think logarithmic scaling of the display would fix it; a pure sine wave tone, even though a speaker and into the mic, generates a full, single bar at the right frequency. So in other words, a test tone works.

But you'll still get noise in most cases, and if it were logrithmic I think it'd go away, because the noise is always in smaller amounts.

But let’s say my raw data peaks range from 0 to 4095. To scale those to a 16-segment display, I divide by 256 which of course works, but that’s a linear conversion (I do it dynamically and adjust gain and more but that’s not relevant here). Being audio/decibels I assume that like a VU meter, it should be logarithmic.

But how do I map 0-4096 down to 0-15 in a logarithmic fashion? I tried a quick pass over the data with something akin to:

Sample = log(Sample*)*
But neither the natural log nor log 10 is really appropriate. Ideally I’d like to hook it up to a potentiometer and adjust the scaling live for the best visuals using music, but I’m not sure how to do an “arbitrary” logarithmic mapping in C.
Any help on how to make a "variably logrithmic" mapping function that I can connect to a pot would be great!

This won't help, but...

I do not have a good anti-aliasing filter as I've not found one in another spectrum analyzer that works!

To avoid aliasing, you MUST NOT allow signals with frequency higher than (sample frequency)/2 to reach the input.

Right, you need an analog filter to filter the signal before it's digitized.

But, aliasing only happens when the signal is above the Nyquist limit (half the sample rate). You won't get aliasing with a sine wave as long as you're below the Nyquist limit.

...Aliasing isn't always a big deal with a "music display" because the highest frequencies (or supersonic frequencies) in music are generally low-level so the aliases are also low level. But, if you are making a spectrum analyzer as a measurement instrument an anti-aliasing filter is critical.

But how do I map 0-4096 down to 0-15 in a logarithmic fashion?

dB = 20 x log (x/Ref)

If 4096 is your zero-dB reference, dB = 20log(x/4096)

...That is log10.

But neither the natural log nor log 10 is really appropriate. Ideally I'd like to hook it up to a potentiometer and adjust the scaling live for the best visuals using music, but I'm not sure how to do an "arbitrary" logarithmic mapping in C.

Any help on how to make a "variably logrithmic" mapping function that I can connect to a pot would be great!

The normal way would simply be a "volume control" pot... That's how just-about everything in audio works... You start-out with plenty of signal and then attenuate with a volume control.

But, you can also do it digitally. Amplification is multiplication. Multiply by 2 to double the signal and multiply by 0.5 to cut the signal in half, etc. After conversion to dB amplification/attenuation is addition/subtraction. Add 6dB to double the signal and subtract 6dB to cut it in half, etc. Or, you can use a different dB reference (something other than 4096).

You could map a pot to change the digital amplification factor or the dB reference, or you can make an automatic sensitivity control...


I've never made a spectrum analyzer but I've made some sound activated lighting effects that work on "loudness". I save a loudness sample* once per second in a 20-second circular buffer.** Then depending on the effect I use the peak or the average from the 20-second buffer as my reference or as my threshold.

I have a "VU meter" effect that is NOT a VU meter. The "top" of the meter is the 20-second peak and the "bottom" of the meter is the 20-second average. (It's not logarithmic. It's a biased-linear scale.) It gives lots of "meter action" with loud or quiet music.

  • This is NOT a waveform sample and it's NOT REALLY "loudness". In my application it's the short-term peak from a peak detector circuit, which is a reasonable approximation of loudness. (You can't use a peak detector with a spectrum analyzer because there is no audio out of it but you can do something similar digitally.)

** See the [u]Smoothnig Example[/u] if you don't know how a circular buffer works.

davepl:
Makes sense but no one else seems to apply a log scale, though I can try it!

A spectrum analyzer is almost always used in logarithmic mode.

I do use windowing, but with the defaults, if that matters.

Yes, most windows are hopeless - what is the default?

I do not have a good anti-aliasing filter as I've not found one in another spectrum analyzer that works!

Its a hardware filter you need, nothing to do with the code.

davepl:
Sample = log(Sample*)*
But neither the natural log nor log 10 is really appropriate. Ideally I’d like to hook it up to a potentiometer and adjust the scaling live for the best visuals using music, but I’m not sure how to do an “arbitrary” logarithmic mapping in C.
Any help on how to make a "variably logrithmic" mapping function that I can connect to a pot would be great!
[/quote]
*_ <em>*Sample[i] = scale * log (Sample[i]) + offset ;*</em> _*

Can you explain the offset?

Speaking of offset, I'm also curious if I need to be removing a DC bias from the analogRead() results?

I really think logarithmic scaling of the display would fix it;

Nope.

A low pass antialiasing filter will help though.

You should remove the DC bias from analogRead. For 10-bit ADC subtract 512 from each sample. For 12-bit ADC subtract 2048.

Pete

davepl:
Can you explain the offset?

Speaking of offset, I'm also curious if I need to be removing a DC bias from the analogRead() results?

Very simple its an offset. So the trace can be where you want vertically. Since this is a logarithmic
value being plotted then only the left most value represents DC.

Remember the output of an FFT is in the frequency domain.

I believe my offset is 1.25V according to the little board labelling. So if 1.25/5.00== 0.25 and I'in in 11bit DAC mode (0-45) it seems like I just remove 0.25*4096 == 1024 that i remove from each analogRead, is that correct?

Failing that I'll write a little app that samples and tells me the average every second, would that work>

11bit DAC mode should give you numbers in the range zero to 2047. If the voltage is offset by 1.25V on a full-scale 5V signal, you'd subtract 512. But the voltage offset should be to the middle of the full-scale signal to get the best resolution. Otherwise you must make sure your input signal is limited to plus or minus 1.25V (2.5V peak to peak).

Pete

I was thinking that I'll know I'm "right" went the sampled data averages to 0.

My theory is that if you back out the DC offset correctly, you've got an AC signal cycling around 0v potential. Doers that make sense, and is it the ideal way to go about it?

If you are taking the spectrum only the zero frequency value in the result depends on the DC offset.

Well, not sure if this is correct, but I wrote a little app to to average to mic analogRead value over time with nothing other than background sound.

The average came out in 508-520 range. I'm using 10 bit sampling right now and the board has a 1.25V DC offset.

So I was surprised the offset was actually HALF of the average sampled volume. So I guess the peaks go 0-1024 naturally or -512 to +512 once adjusted. That actually makes a bit of sense to me, though I can't say its right.