Semi-advanced Averaging

Hallo All.

So I have a little issue with my hardware and I would like to try and solve this problem with software...

I have an external 12bit A/D converter. unfortunately the signal it receives has a bit of noise on it with some crazy spikes.

data example: 250 255 245 249 3000 240 250 254 4123 250 [end]

How I want to solve this is to receive 10 values and remove the ones that are way out from the previous ones and then Average what is left. The big issue I am struggling with is my code has to chose a value as its yard stick, now how would I stop it from choosing one of the noise spikes as its yard stick...

Any help or guidance would be appreciated. Thank you.

Would a moving average help? Possibly refraining from using the result until you have enough samples to make the average smooth.

Thank you for the response wildbill.. I am currently trying to do something similar to that but using digital data. The A/D converter I am using is external and gives digital values to the Arduino.

I'm thinking of something like this:

MovingAverage=(MovingAverage*19+ADCReading)/20;

Details of there the ADC reading came from shouldn't matter. In this case, process twenty (or more) readings before you use the average for anything important. Even if you start the average with a spike, the effect will be substantially reduced here.

I understood it wrong then. So I am going to try implement it as follows

if(((previousSave+25) >= ADCReading) && ((previousSave-25) <= ADCReading)){
MovingAverage=(MovingAverage*19+ADCReading)/20;
}

Just to single out the spikes…

But a 10x spike will still really change the numbers. the average of 19 250's and 1 3000 is still 388, which (assuming that the 3000 should be ignored) is way out of line.

I suggest a running measurent of Standard Deviation and any point two or 3 standard deviations away is ignored.

I just did a test where I generated 20 numbers evenly distributed between 245 and 255.

The average for those 20 numbers was 249 and the standard deviation was about 3. I threw a 3000 into the mix and the average for those was 374 and the standard deviation was 590.

So, even if I set a 3 sd limit for the data, in the first case I would accept everything from 241 to 259, or every data point. In the second case I would accept everything from -1400 to 2144, eliminating the point at 3000. Note that this applies to the first dataset, Future datasets can benefit from the known standard deviation based on the filtered data.

So I managed to achieve something…

Here is my code if anyone is interested… No comments though.

int num[16];
int averageArray[10];
int reggie = 0;
int previousSave = 0;
int x,y = 0;
int average = 0;
int noSave = 0;
int fail = 0;

void setup(){
  num[0] = 654;
  num[1] = 255;
  num[2] = 245;
  num[3] = 249;
  num[4] = 250;
  num[5] = 3000;
  num[6] = 250; 
  num[7] = 254;
  num[8] = 4123;
  num[9] = 250;
  num[10] = 234;
  num[11] = 240;
  num[12] = 250;
  num[13] = 250;
  num[14] = 256;
  num[15] = 270;
  
  Serial.begin(9600);
}

void loop(){
    
  start:
  if(noSave != 1){
    previousSave = num[x];
  }
    
  while(reggie <= 9){
    
    if(((previousSave+25) >= num[x]) && ((previousSave-25) <= num[x])){
      noSave = 1;
      previousSave = num[x];
      averageArray[y] = num[x];
      Serial.print(y);
      Serial.print(" saved = ");
      Serial.println(averageArray[y]);
      y++;
      reggie++;
    }
    
    else{
      fail++;
      Serial.println("failed");
      if(fail > 3){
        y = 0;
        reggie = 0;
        fail = 0;
        noSave = 0;
        goto start;
      }
    }
    x++;
  }
  noSave = 0;
  
  for(int sum = 0; sum <= 9; sum++){
    average = average + averageArray[sum];
  }
  average = average/10;
  
  Serial.println(average);
  delay(90000);
}

Thanks for the help