DC removal

Projects Programming
1

Hi
I have a program to remove DC offset from signal, it is working OK itself, when I added it to another program a big delay occurred, over 15 sec. After uploading the sketch serial plotter is dead for 30 sec. then signal is appearing. When I turn signal ON/OFF the blue line is responding instantly (input signal) from 0 to 70 000 on scale but DC removed signal is changing slowly (red line), to get to 0 it is take over 15 sec.
It is possible to fix it ?

#include "arduinoFFT.h"

#define SAMPLES1 128             //Must be a power of 2
#define SAMPLING_FREQUENCY 40000 //Hz, must be less than 10000 due to ADC

arduinoFFT FFT = arduinoFFT();

unsigned int sampling_period_us;
unsigned long microseconds;

double vReal[SAMPLES1];
double vImag[SAMPLES1];
///////////////////////////////

int sample = vReal[40];
//int sample = 0;
int last_sample = 0;
//long shifted_filter = -10000;
long shifted_filter = -1;
//////////////////////////////
void setup() {
  Serial.begin(115200);

  sampling_period_us = round(1000000 * (1.0 / SAMPLING_FREQUENCY));
}

void loop() {
  /*
  /////////////////////////
  last_sample = sample;
  sample = vReal[40];

 long shiftedFCL = shifted_filter + (long)((sample - last_sample) << 8);
  //long shiftedFCL = shifted_filter + (long)((vReal[40] - last_sample) << 8);
  shifted_filter = shiftedFCL - (shiftedFCL >> 8);
  long filtered_value = (shifted_filter + 128) >> 8;
  ///////////////////////////
  */
  /*SAMPLING*/
  for (int i = 0; i < SAMPLES1; i++)
  {
    microseconds = micros();    //Overflows after around 70 minutes!

    vReal[i] = analogRead(PA7);
    vImag[i] = 0;

    while (micros() < (microseconds + sampling_period_us)) {
    }
  }

  /*FFT*/
  FFT.Windowing(vReal, SAMPLES1, FFT_WIN_TYP_HAMMING, FFT_FORWARD);
  FFT.Compute(vReal, vImag, SAMPLES1, FFT_FORWARD);
  FFT.ComplexToMagnitude(vReal, vImag, SAMPLES1);
  double peak = FFT.MajorPeak(vReal, SAMPLES1, SAMPLING_FREQUENCY);
///////////////////////////////
  last_sample = sample;
  sample = vReal[40];

 long shiftedFCL = shifted_filter + (long)((sample - last_sample) << 8);
  //long shiftedFCL = shifted_filter + (long)((vReal[40] - last_sample) << 8);
  shifted_filter = shiftedFCL - (shiftedFCL >> 8);
  long filtered_value = (shifted_filter + 128) >> 8;
///////////////////////////////
  /*PRINT RESULTS*/
  Serial.print(vReal[40]); //blue
  /////////////////
  //Serial.print(' ');
  //Serial.print(sample); //red
  Serial.print(' ');
  Serial.println(filtered_value);// green
 ////////////////
}

15 sec.jpg
15 sec.jpg570×540 29.6 KB

2

pic

15 sec.jpg
15 sec.jpg570×540 29.6 KB

3

I guess I would Serial.println( sampling_period_us);

See if the value is what you want from this line:

sampling_period_us = round(1000000 * (1.0 / SAMPLING_FREQUENCY));

Paul

4

How long does it take to run that FFT calculation? I'm guessing that's what's slowing you down.

5 
#define SAMPLING_FREQUENCY 40000 //Hz, must be less than 10000 due to ADC

? 40000 must be less than 10000?
hmmm

6

The adding for DC removal are between
forward slash lines, sampling period us .... belong to blue line, which is working OK.
"I guess I would Serial.println( sampling_period_us); " not shure how to use it, put it on the end ?

7

demkat1:

#define SAMPLING_FREQUENCY 40000 //Hz, must be less than 10000 due to ADC

? 40000 must be less than 10000?
hmmm

That is leftover from another program.

8

Jiggy-Ninja:
How long does it take to run that FFT calculation? I'm guessing that's what's slowing you down.

FFT it is a blue line and it is working fine. If FFT will cause the problem then delay should be on both lines.

9

Vik321:
Hi
I have a program to remove DC offset from signal, it is working OK itself, when I added it to another program a big delay occurred, over 15 sec. After uploading the sketch serial plotter is dead for 30 sec. then signal is appearing.

Emulated floating point FFT will be glacially slow. This is to be expected. FFT on the ATmega based Arduinos
should be done in fixpoint, not floating point.

When I turn signal ON/OFF the blue line is responding instantly (input signal) from 0 to 70 000 on scale but DC removed signal is changing slowly (red line), to get to 0 it is take over 15 sec.

You have a high pass filter, not DC removal. DC removal is done after the samples are collected and is an operation on the entire set (or alternatively by pre-calibrating the DC zero level so it can be subtracted out of every sample) - you have a digital high-pass filter with a roll-off frequency of about 0.4% Fs

It is possible to fix it ?

Find a better FFT library with fixpoint option?
Do DC removal differently?

10

Vik321:
FFT it is a blue line and it is working fine. If FFT will cause the problem then delay should be on both lines.

Nope. Your "DC removal" code is basically a discrete high-pass filter. Its time response is heavily dependent on its sampling rate. If you slow down the sampling rate, it takes a longer amount of time to equalize. And trust me, adding in that FFT code that takes 128 samples every loop is going to cause an extreme reduction in whatever sampling rate you had before. And with your damping coefficient so close to 1 (looks like it's effectively 255/256), a slow sampling rate will take forever reach equilibrium.

11

Thanks for explanation, now I understand the problem.
I like this FFT, so I have to find a different way for DC removing, without filters.
The best will be code which do the same as capacitor ?

12

Yes.

13

Vik321:
Thanks for explanation, now I understand the problem.
I like this FFT, so I have to find a different way for DC removing, without filters.
The best will be code which do the same as capacitor ?

That is what your code already does. That's what "discrete high-pass filter" means. You don't need to throw out the code you already have, just adjust the damping constant of your filter so that it can respond properly for our new sampling time.

 long shiftedFCL = shifted_filter + (long)((sample - last_sample) << 8);
  shifted_filter = shiftedFCL - (shiftedFCL >> 8);
  long filtered_value = (shifted_filter + 128) >> 8;

These expressions appear to be an "optimized" version of this equation for a discrete high-pass filter:

filter_value = damping_coefficient * (filter_value + new_reading - previous_input);

I use "optimized" in scare quotes because while the implementation you are using will be more efficient to calculate, it is also more obscure, making it harder to understand and tune. Your second line is subtracting 1/256th of a value from itself, which is mathematically equivalent to multiplying that value by 255/256. That is the damping coefficient this code uses. Unfortunately, this optimization can only give you specific values for the damping coefficient, which will probably not be good for this filter. You should be able to replace that line with shifted_filter = shiftedFCL * damping_coefficient;. Note that the damping coefficient will always be between 0 and 1, so you might have to split it up to multiply by the numerator then divide by the denominator, or change it to use floating point or fixed point numbers.

To get the correct damping value, you need two numbers: you need to measure the actual amount of time your program takes between two invocations of this filter, and you need choose how much time you want the filter to take to get to 0 (then divide that time by 5 to get your filter's time constant).

Once you have the actual sampling period and time constant, you can calculate the damping coefficient with this equation: TimeConstant / (TimeConstant + SamplePeriod).

With your effective damping coefficient being so close to 1, your filter value decays very slowly. This equation will probably give you something closer to 0 and speed up the decay (which is what you want).

14

Thanks
This is what I did, can you guide me to implement your suggestions ?

#include "arduinoFFT.h"

#define SAMPLES1 128             //Must be a power of 2
#define SAMPLING_FREQUENCY 40000 //Hz, must be less than 10000 due to ADC

arduinoFFT FFT = arduinoFFT();

unsigned int sampling_period_us;
unsigned long microseconds;

double vReal[SAMPLES1];
double vImag[SAMPLES1];
///////////////////////////////
int damping_coefficient = 0.5; // between 0 and 1
int filter_value;
int new_reading;
int previous_input;

int sample = vReal[40];
//int sample = 0;
int last_sample = 0;
//long shifted_filter = -10000;
long shifted_filter = -1;
//////////////////////////////
void setup() {
  Serial.begin(115200);

  sampling_period_us = round(1000000 * (1.0 / SAMPLING_FREQUENCY));
}

void loop() {

  /*SAMPLING*/
  for (int i = 0; i < SAMPLES1; i++)
  {
    microseconds = micros();    //Overflows after around 70 minutes!

    vReal[i] = analogRead(PA7);
    vImag[i] = 0;

    while (micros() < (microseconds + sampling_period_us)) {
    }
  }

  /*FFT*/
  FFT.Windowing(vReal, SAMPLES1, FFT_WIN_TYP_HAMMING, FFT_FORWARD);
  FFT.Compute(vReal, vImag, SAMPLES1, FFT_FORWARD);
  FFT.ComplexToMagnitude(vReal, vImag, SAMPLES1);
  double peak = FFT.MajorPeak(vReal, SAMPLES1, SAMPLING_FREQUENCY);
  ///////////////////////////////
  last_sample = sample;
  sample = vReal[40];
  /*
    long shiftedFCL = shifted_filter + (long)((sample - last_sample) << 8);
    //long shiftedFCL = shifted_filter + (long)((vReal[40] - last_sample) << 8);
    shifted_filter = shiftedFCL - (shiftedFCL >> 8);
    long filtered_value = (shifted_filter + 128) >> 8;
    //////////////////////////////////////////////////////////////////
  */
 
  long shiftedFCL = shifted_filter + (long)((sample - last_sample) << 8);

  shifted_filter = shiftedFCL * damping_coefficient;
  filter_value = damping_coefficient * (filter_value + new_reading - previous_input);
  ////////////////////////////////////////////////////////////////
 
  /*PRINT RESULTS*/
  Serial.print(vReal[40]); //blue
  /////////////////
  //Serial.print(' ');
  //Serial.print(sample); //red
  Serial.print(' ');
 // Serial.println(filtered_value);// green
  Serial.println(filter_value);// 
  
  ////////////////
}
15 
   while (micros() < (microseconds + sampling_period_us)) {

While the statement below is mathematically equivalent in the maths you were taught at school, it is not equivalent inside the Arduino. By using subtraction instead of addition, it eliminates the rollover error.

    while (micros() - microseconds < sampling_period_us) {

Nick Gammon has a great webpage describing the rollover issues in millis() and micros().

Your code will "creep" the sampling period. By the time you get back to the top of the loop and update your timing variable, the value of micros() will have changed. For something like this where you want a good reliable interval, I always add instead of querying micros() again.

     microseconds += sampling_period_us;
16

Thanks
I will do this, but first I need implement Ninja suggestions, right now

Serial.println(filter_value);

prints 0

17

Make it a float.

18

removed

int filter_value;

added

void loop() {
float filter_value;

no changes

19

Where does the code put values into new_reading and previous_input?

20

Good point
I just put suggestions into code and eliminate compiling errors now I have to analyze it