RunningAverage Class on playground (updated)

updated Running Average Library to version 0.2.05 on GITHUB

changed float to double
==> support for double on ARM of the Arduino DUE …

Thanks Rob.

BTW I'm using a MEGA with GPS engines running at 4Hz...

updated Running Average Library to version 0.2.06 on GITHUB

skip version 0.2.07 (contained typos)

updated Running Average Library to version 0.2.08 on GITHUB - https://github.com/RobTillaart/Arduino/tree/master/libraries/RunningAverage -

since 0.2.06 + added getMin() and getMax() - thanks to Eric M. + refactored

Note getMin() and getMax() gives the minimum/maximum since last call to clear(), ==> so it is not the min/max of the internal buffer.

Do you have any plans on getting your libraries added to the 1.6.2+ library manager? I imagine it would take quite a bit of work because it seems at first brush that each library should be it's own repository (instead of how you have several libraries in one repository).

I think AdaFruit has published a script to help mass publishing of libraries.

No I do not have such plans in near future as my time is quite limited.

And I am using 1.5.8 which works quite well for me.

robtillaart: No I do not have such plans in near future as my time is quite limited.

And I am using 1.5.8 which works quite well for me.

OK. Fair answer.

One thing to be wary of when using this library is floating point imprecision.

I left a unit running for a week sampling 10 times a second and was very confused when I came back as to why my average (over the last 50 samples) was very different to the actual current input.

Looking into the code, the RunningAverage::addValue function subtracts a Double and then adds a Double. Adding and subtracting of Doubles (and Floats) can lead to a huge degree of imprecision.

I'm not sure what the best solution is here, calculating the sum total every time getAverage() is called is the obvious one, but that leads to somewhat more function overhead.

You are completely right. Thanks for this observation.

10 samples/second = 864.000 /day = 6 million per week.

The number of samples is approaching the accuracy of the (IEEE754) float (7 digits) when one reaches 1 million samples (1 day in your case) . This means if the average sample has 3 digits, one will loose 2 digits every time.

The "trick" of remove one double and add one double is an efficient way to calculate the sum of the array of samples. And yes when using this trick the error will add up in sum. In theory after 100 samples the last two digits are suspected to be incorrect. Need to investigate the effect.

Solution is to (sort and) add all elements of the array every time, giving a constant error. Consequence is a slower execution. The sorting will improve the sum when the dynamic range of the numbers is large.

A practical solution for the library might be to do a proper sum e.g. every 100 or 1000 times a value is added (new parameter?). The lower this number the more accurate the sum. The price is a performance penalty once every 100 / 1000 times. There are several variations possible. A good value for the number will also depend on the size of the numbers added. Ideally one would like to have a sort of error counter that makes an estimate of the max error and when a threshold is reached do a recalculation of sum.

The easiest solution is to redo the function Average() and rename the current one to fastAverage();

in short food for thought.

thanks

quick patch:

// returns the average of the data-set added sofar
double RunningAverage::getAverage() const
{
    if (_cnt == 0) return NAN;
    double sum = 0;
    for (uint8_t i = 0; i < _cnt; i++)
    {
        sum += _ar[i];
    }
    return sum / _cnt;
}

double RunningAverage::getFastAverage() const
{
    if (_cnt == 0) return NAN;
    return _sum / _cnt;
}

the .h file must add a signature for

double getFastAverage() const;

updated Running Average Library to version 0.2.11 on GITHUB https://github.com/RobTillaart/Arduino/tree/master/libraries/RunningAverage -

The following changes were made since 0.2.08 (request aaronblanchard) + getAverage() renamed to getFastAverage() as it is fast but less accurate. + reimplemented getAverage() to be accurate, but a bit slower. getAverage() sums all elements in the internal buffer and average them. This solves the remark from aaronblanchard a few posts ago that the fastAverage() drifts from the real value. An example sketch is included to show this drift.

(request by mail) + added GetMinInBuffer() to make difference with getMin() + added GetMaxInBuffer() to make difference with getMin() Note getMin() and getMax() gives the minimum/maximum since last call to clear(). An example sketch shows the differences between the methods.

  • refactored

snippet from drift test (included as example)

COUNT       AVG          FASTAVG     DIFF        MAXDIFF sofar
30750000    0.5153000   0.5152977   0.0000023   0.0000522
30760000    0.5241000   0.5240974   0.0000026   0.0000522

The sample sketch adds values between 0.0 and 0.999 in a loop. The maximum difference after 30 million++ additions is relative still low. There will be input streams which are more sensitive for the fastAverage algorithm. So when max accuracy is a requirement please use getAverage().

as always remarks and bug reports are welcome, Rob

robtillaart: updated Running Average Library to version 0.2.11 on GITHUB https://github.com/RobTillaart/Arduino/tree/master/libraries/RunningAverage -

Maybe a stupid question but where is the download button for this library? I must be missing something. :-[

No stupid question, the [download zip] button is two levels up (right side)

https://github.com/RobTillaart/Arduino

or use

https://github.com/RobTillaart/Arduino/archive/master.zip

robtillaart: No stupid question, the [download zip] button is two levels up (right side)

https://github.com/RobTillaart/Arduino

OK, thanks, sorry that I missed that. I was too deep. ;)

There are tricks you can use to improve accuracy when adding up a lot of floats.
For example, you can use a long to hold the “whole” part of the sum, and a float to hold the fractional part.

Good point, it’s a variation of - Kahan summation algorithm - Wikipedia

that said accuracy tricks often come at a price of footprint, performance or sometimes the working range of the code. The more you know about the data to process the easier/better you can tune an algorithm for accuracy/performance etc.

hi!

First: Thanks for publishing your code, great work!

I'm using it to filter the input of opto sensors for reading the speed of two dc motors, but since the speed of the motors is variable the cutoff of the filter should too, exist a way to change the amount of samples in the running average on the fly?

Can I run 3 copies of RunningAverage.h at the same time? I am attempting to find relative accelerometer motion in three axis simultaneously to produce a z score. this is a test using example furnished with the library. Can I interleave or multiplex. Sample size will probably need tobe altered on thefly as code progresses. Thanks a lot.

// // FILE: runningAverageTest.pde // AUTHOR: Rob Tillaart // VERSION: 0.1.01 // DATE: 2012-12-30 // // PUPROSE: show working of runningAverage //

include "RunningAverage.h"

RunningAverage myRA(10); int samples = 0;

void setup(void) { Serial.begin(115200); Serial.println("Demo RunningAverage lib"); Serial.print("Version: "); Serial.println(RUNNINGAVERAGE_LIB_VERSION); myRA.clear(); // explicitly start clean }

void loop(void) { int x=analogRead(A0); int u=myRA.getAverage(); int s=myRA.GetStandardDeviation();

int z=(x-u)/s; //long rn = random(0, 1000); myRA.addValue(x); // samples++; // Serial.print(samples); // Serial.print("\t Running Average: "); Serial.println(z);

/* if (samples == 300) { samples = 0; myRA.clear(); Serial.println(); }*/ delay(10); }

Yes you can by allocating 3 running Average objects.

Add the X values to the RA_X object, the Y values to the RA_Y object etc

see small sketch below

//
//    FILE: runningAverage.ino
//  AUTHOR: Rob Tillaart
// VERSION: 0.0.1
//    DATE: 2017-07-24
// PURPOSE: demo
//
// HISTORY:
// 0.0.1 - 2017-07-24 initial version

#include "RunningAverage.h"

RunningAverage RA_X(10);
RunningAverage RA_Y(10);
RunningAverage RA_Z(10);

void setup(void)
{
  Serial.begin(115200);
  Serial.println("Demo RunningAverage lib");
  Serial.print("Version: ");
  Serial.println(RUNNINGAVERAGE_LIB_VERSION);

  RA_X.clear();
  RA_Y.clear();
  RA_Z.clear();
}

void loop(void)
{
  int x = analogRead(A0);
  int y = analogRead(A1);
  int z = analogRead(A2);

  RA_X.addValue(x);
  RA_Y.addValue(y);
  RA_Z.addValue(z);

  Serial.print("RA_X: \t");
  Serial.println(RA_X.getAverage());
  Serial.println(RA_X.GetStandardDeviation());

  // idem for RA_Y and RA_Z

  int u = RA_X.getAverage();
  int s = RA_X.GetStandardDeviation();
  int zzz = (x - u) / s;

  Serial.print("ZZZ:\t");
  Serial.println(zzz);

  delay(10);
}

Is it possible to only clear Min/Max values?