RunningAverage Class on playground (updated)

A question in this forum from Liudr - http://arduino.cc/forum/index.php/topic,50383.0.html - triggered me to implement a small runningAverage class. This can be found on - Arduino Playground - RunningAverage . A sample sketch shows how one can create running average on 2 different levels, - last minute, last hour - when one has a sample per second. In the sketch the sensor is simulated by the random function and samples are made far faster than one per second but in sample code this is allowed I guess :slight_smile:

Please post comments and improvements in this thread,

thanks,
Rob

//
//    FILE: runningAverageHour.pde
//  AUTHOR: Rob Tillaart
//    DATE: 2012-12-30
//
// PUPROSE: show working of runningAverage per hour 
//          in 2 steps - last minute + last hour
//          3 or more steps also possible
//

#include "RunningAverage.h"

RunningAverage raMinute(60);
RunningAverage raHour(60);

int samples = 0;

void setup(void) 
{
  Serial.begin(115200);
  Serial.println("Demo RunningAverage lib - average per minute & hour");
  Serial.print("Version: ");
  Serial.println(RUNNINGAVERAGE_LIB_VERSION);
  raHour.clear();
  raMinute.clear();
}

void loop(void) 
{
  long rn = random(0, 100);
  raMinute.addValue(rn);
  samples++;
  
  if (samples % 60 == 0) raHour.addValue(raMinute.getAverage());
  
  Serial.print("  raMinute: ");
  Serial.print(raMinute.getAverage(), 4);
  Serial.print("  raHour: ");
  Serial.println(raHour.getAverage(), 4);
}

updated example to reflect 0.2.02 version

Thanks a lot Rob. The sample code here is very nice. I love the two level average, nice for OOP. I guess I can set the sample amount to 60 or other quantity? I'll look into it later today!

Thanks for that, it'll save me writing something for an upcoming project. I'll play with it when the time comes.


Rob

Rob, Just tried to get your example to go but comes up with a compile error.

\libraries\RunningAverage\RunningAverage.cpp:26: error: definition of implicitly-declared 'RunningAverage::~RunningAverage()'

Any help appreciated .

Thanks Rock.

Which version of Arduino are you using?
- short term patch is to comment the ~RunningAverage() from .cpp and .h but that would create a memory leak. However as long as the object is in scope and there is no need to free the internal array that should not give a problem.
I'll have to dive in this one

update:

The .h file is missing a prototype of the destructor. Add ~RunningAverage(); in the public section just under the constructor. That should fix it.
Playground article is updated.

Thanks for finding this one.
Rob

Rob, Thanks for that, the compiler gets past that, but now comes up with another problem, error : request for member 'clr' in 'myRA', which is of non-class type 'RunningAverage ()()'.
This occurs also for member 'add' and 'avg' when I comment out the other problem lines.

I know zip about C++ so I'm at a loss .

Thanks Rock.

Please post your sketch so I can check this one too.

Rob, I found the problem , I had not deleted the .o file and restarted the IDE after changing the .h file.

compiles OK now.

Thanks for your help.

Rock.

Found a newer versio of the runningAverage class by Yuval Naveh here

most important addition:

void RunningAverage::trimToValue(float value)
{
	clr();
	add(value);
}

TODO: update the playground

Finally found some time to update the runningAverage lib on the playground: - Arduino Playground - HomePage -

changes:

  1. There are some breaking changes in the method names to make them more descriptive;
    // clr() => clear()
    // add(x) => addValue(x)
    // avg() => getAverage()

  2. new is the fillValue() function, based upon the trimValue() of Yuval Naveh - see previous post

// fill the average with a value
// the param number determines how often value is added (weight)
// number should preferably be between 1 and size
void RunningAverage::fillValue(float value, int number)
{
	clear();
	for (int i = 0; i < number; i++) 
	{
		addValue(value);
	}
}

It fills the internal array with a number of identical values to get a starting value for average.
By adding more than one value, the initial average gets a certain weight.
This extends the original trimValue() so that's why I gave another name.

  1. some small refactoring and added comments

as always comments, remarks and ideas are welcome

Hi there,

I´m using the runningAverage library and got some problems when clearing the averaging variabel every 300 counts.
Now, I did a Measurement series showing my raw sensor data and the avaregd data (see attached Differenzdruck_gemittelt_100.pdf). Looking at these curves, it is clearly visible that my avareged sensor data is going bogus when the averaging variabel is cleared after 300 samples. Now, looking at the library I found the option to use the a "fillvalue" instead. As the code is not that good commented, I just thought to ask what it actually does. However, Im loosing the peaks in my sensor data but seems to oscillate now. Well, to make a long story short, could someone explain the fillvalue feature of the running average library in more detail.

Code using the running average (plain and simple)

/* 
 ************************************* 
 * Thread: THREAD_DifferentialPressure
 * Get`s the differential pressure.
 ************************************* 
 */

/* Global variables for this function */
float diff_Pressure = 0;
float offset_diff_pressure;              // offset which can be set by the user -> FUNC_OffsetDiffPressure (X_E_LCDML_MenuFunctions) 
RunningAverage diff_p_RA(100);            // use default size for running average for differential pressure
int diff_p_samples = 0;                  // create a samples counter
  /* --------- INIT ---------
     * Initialization of this function
     * is only being done once at the start of the function
     */
 void simpleThread_setup(THREAD_DifferentialPressure)
  {   
  /* Init Function */
  diff_p_RA.clear();                     // Running Average for diff. pressure; explicitly start clean
  }
     
  /* --------- LOOP ----------
     * This is the place for code which is going to be repeated constantly 
     * e.g. a time-diplay or the like.
     */
 boolean simpleThread_loop(THREAD_DifferentialPressure)
  {
  /* Loop Function */
  diff_p_RA.addValue(getDifferentialPressure()); // read out the sensor value and add to averaging variable
  diff_p_samples++;                      // increase samples counter
  // set averaged differential pressure  - pressure offset set by the user
  diff_Pressure = diff_p_RA.getAverage() - offset_diff_pressure;
  Serial.println(diff_Pressure);
  if (diff_p_samples == 300)             // handle large samplenumbers...
  {
    diff_p_samples = 0;                // ...delete samples counter, and...
    diff_p_RA.clear();                 // ...clear the averaging variable.
  }
  else if (isnan(diff_Pressure) || diff_Pressure < 0.00)
  {
    diff_Pressure = 0.00;    // set diff_Pressure to zero if the reading is NaN - Not a Number, or lower than zero
  }
  return true;
  }

Same code using the running average wih the fill value option:

/* 
 ************************************* 
 * Thread: THREAD_DifferentialPressure
 * Get`s the differential pressure.
 ************************************* 
 */

/* Global variables for this function */
float diff_Pressure = 0;
float offset_diff_pressure;              // offset which can be set by the user -> FUNC_OffsetDiffPressure (X_E_LCDML_MenuFunctions) 
RunningAverage diff_p_RA(100);            // use default size for running average for differential pressure
int diff_p_samples = 0;                  // create a samples counter
  /* --------- INIT ---------
     * Initialization of this function
     * is only being done once at the start of the function
     */
 void simpleThread_setup(THREAD_DifferentialPressure)
  {   
  /* Init Function */
  diff_p_RA.fillValue(0,50);                     // Running Average for diff. pressure; explicitly start clean
  }
     
  /* --------- LOOP ----------
     * This is the place for code which is going to be repeated constantly 
     * e.g. a time-diplay or the like.
     */
 boolean simpleThread_loop(THREAD_DifferentialPressure)
  {
  /* Loop Function */
  diff_p_RA.addValue(getDifferentialPressure()); // read out the sensor value and add to averaging variable
  diff_p_samples++;                      // increase samples counter
  // set averaged differential pressure  - pressure offset set by the user
  diff_Pressure = diff_p_RA.getAverage() - offset_diff_pressure;
  if (diff_p_samples == 300)             // handle large samplenumbers...
  {
    diff_p_samples = 0;                  // ...delete samples counter, and...
    diff_p_RA.fillValue(diff_Pressure, 25); // ...clear the averaging variable.
  }
  else if (isnan(diff_Pressure) || diff_Pressure < 0.00)
  {
    diff_Pressure = 0.00;    // set diff_Pressure to zero if the reading is NaN - Not a Number, or lower than zero
  }
  return true;
  }

Thank you for any help.
Best regards,
Jan

Differenzdruck_gemittelt_100.pdf (182 KB)

Differenzdruck_gemittelt_100_filled.pdf (181 KB)

ANSWER BY ROB TILLAART (send by e-mail):

In the example (of the RunningAverage library) I am clearing the RA variable every 300 count so people see how it can be done and what its effect is on the RA. And that effect is exactly what you see in the RA, the smooth line breaks. The number 300 is just arbitrary.

There can be several reasons why one wants to clear the RA var.
(1) one starts a (really) new series of measurements, Suppose one makes a RA measurement of the light-intensity for one minute once per hour. Then you really want to start clean to make the measurements independent.
(2) when you switch to another range of your input , e.g. you switch the analogReference of a VoltMeter sketch from 5.0 to 1.1Volt.
(3) when you switch the timing between the values, if you have a series with a 1 second interval and you change the interval to 1 minute you must clear the RA otherwise you will compare apples with pears.
(4) when the value measured changes more than a certain value (jumps to another level) one might want to follow immediately.
In the PDF's on the forum post (see link above) there is such a jump around 500 and 900.

That said, in most sketches that run continuously do not need to clear/reset the RA. it can go on and on and on forever.

On the forum there is also the question about fillValue(value, number);
This function can be used to initialize the RA on a certain value with a certain weight,
Example (assume buffersize = 10
if you do a clear() followed by a addValue(100); the RA will be 100. another addValue(50) will set the RA to 75.0.
if you do a fillValue(100, 10), followed by a addValue(100); the RA will be 100, another addValue(50) will set the RA to 95.0,
if you do a fillValue(100, 5), followed by a addValue(100); the RA will be 100, another addValue(50) will set the RA to 92,8571...,

So fillValue has its use especially at the start of a measurement

Hi Rob,

thanks for the help! Another thing I was stumbling upon was the question if there were any possible chances of creating an RA-array:

RunningAverage diff_p_RA(50);

in a way that the size (int) would be user configurable during runtime? Now, as I understand it, there is quite a fuz about growing or reducing an array during runtime - meaning that it is not that simple. However, maybe you have an idea for an approach one could try implementing. Maybe, using a fixed large array of the maximum expected size (200 would be a reasonable value -I guess) and then only using parts of that array - as the user defines the averaging to be done between 0 and 200 values.

Best regards,
Jan

Hi Rob,

I did some more measurements and used RunningMedian instead of RunningAverage. The Code I used can be found here:

/* 
 ************************************* 
 * Thread: THREAD_DifferentialPressure
 * Get`s the differential pressure.
 ************************************* 
 */

/* Global variables for this function */
float diff_Pressure_median = 0;
float diff_Pressure = 0;
float offset_diff_pressure;     // offset which can be set by the user -> FUNC_OffsetDiffPressure (X_E_LCDML_MenuFunctions)
int n_averaging;                // variable can be set by the user to define ow many measurements shall be averaged 
RunningMedian diff_p_RA(200);   // use a uer defined size for the running averaging of the differential pressure
  /* --------- INIT ---------
     * Initialization of this function
     * is only being done once at the start of the function
     */
 void simpleThread_setup(THREAD_DifferentialPressure)
  {   
  /* Init Function */
  //diff_p_RA.clear;              // Running Average for diff. pressure; explicitly start clean
  }
     
  /* --------- LOOP ----------
     * This is the place for code which is going to be repeated constantly 
     * e.g. a time-diplay or the like.
     */
 boolean simpleThread_loop(THREAD_DifferentialPressure)
  {
  /* Loop Function */
  diff_p_RA.add(getDifferentialPressure()); // read out the sensor value and add to averaging variable
  // set averaged differential pressure  - pressure offset set by the user
  diff_Pressure = diff_p_RA.getAverage() - offset_diff_pressure;
  diff_Pressure_median = diff_p_RA.getMedian() - offset_diff_pressure;
  Serial.print(diff_Pressure);
  Serial.print(";");
  Serial.println(diff_Pressure_median);
  if (isnan(diff_Pressure) || diff_Pressure < 0.00)
  {
    diff_Pressure = 0.00;    // set diff_Pressure to zero if the reading is NaN - Not a Number, or lower than zero
  }
  return true;
  }

The results of these measurements can be found in the attached pdf file "Diff_Pressure_averaged_median.pdf". Now there is something strange about the averaging value. The buffer size seems not to change. even the 200 value buffer size ("RunningMedian diff_p_RA(200); ") looks more like the 25 or even 10. The RunningMedian Code I used is from your (Rob Tillaart) github repository. Furthermore, I expected something else from the RunningMedian. Let me know what you think.

Best regards,
Jan

P.S.: I now added the raw data of these measurements.

Diff_Pressure_averaged_median.pdf (893 KB)

Diff_Pressure_averaged_median.zip (2.46 MB)

Hi there,

just to complete the statements above, I made another series of measurements (see "Diff_Pressure_RunningAverage.pdf") using the same buffer sizes. But now I used RunningAverage again. As can be clearly seen, the output is as expected. I did not clear the variable as this is not necessary with these measurements.
so one may draw the conclusion that there is something wrong with the code of the RunningMedian Library.

Here is the code sniplet where I calculate my RunningAverage for the differential pressure:

/* 
 ************************************* 
 * Thread: THREAD_DifferentialPressure
 * Get`s the differential pressure.
 ************************************* 
 */

/* Global variables for this function */
float diff_Pressure_median = 0;
float diff_Pressure = 0;
float offset_diff_pressure;     // offset which can be set by the user -> FUNC_OffsetDiffPressure (X_E_LCDML_MenuFunctions)
int n_averaging;                // variable can be set by the user to define ow many measurements shall be averaged 
RunningAverage diff_p_RA(200);   // use a uer defined size for the running averaging of the differential pressure
  /* --------- INIT ---------
     * Initialization of this function
     * is only being done once at the start of the function
     */
 void simpleThread_setup(THREAD_DifferentialPressure)
  {   
  /* Init Function */
  diff_p_RA.clear();              // Running Average for diff. pressure; explicitly start clean
  }
     
  /* --------- LOOP ----------
     * This is the place for code which is going to be repeated constantly 
     * e.g. a time-diplay or the like.
     */
 boolean simpleThread_loop(THREAD_DifferentialPressure)
  {
  /* Loop Function */
  diff_p_RA.addValue(getDifferentialPressure()); // read out the sensor value and add to averaging variable
  // set averaged differential pressure  - pressure offset set by the user
  diff_Pressure = diff_p_RA.getAverage() - offset_diff_pressure;
  Serial.println(diff_Pressure);
  if (isnan(diff_Pressure) || diff_Pressure < 0.00)
  {
    diff_Pressure = 0.00;    // set diff_Pressure to zero if the reading is NaN - Not a Number, or lower than zero
  }
  return true;
  }

Best regards,
Jan

P.S.: I now added the raw data of these measurements.

Diff_Pressure_RunningAverage.pdf (858 KB)

Diff_Pressure_RunningAverage.zip (2.94 MB)

Take a look at my Library, that works perfecly for a moving average...

jabami:
[... snipped ...]
The results of these measurements can be found in the attached pdf file "Diff_Pressure_averaged_median.pdf". Now there is something strange about the averaging value. The buffer size seems not to change. even the 200 value buffer size ("RunningMedian diff_p_RA(200); ") looks more like the 25 or even 10. The RunningMedian Code I used is from your (Rob Tillaart) github repository. Furthermore, I expected something else from the RunningMedian. Let me know what you think.

Best regards,
Jan

P.S.: I now added the raw data of these measurements.

If you look in RunningMedian.h you will see

#define MEDIAN_MAX_SIZE     19          // adjust if needed

If you don't change that, requesting a median array size of anything larger than 19 will only give you a 19 element array to run your medians and averages against. Did you change that line before compiling your sketch that requests a 200 element array?

Hi there,

thank you Sembazuru for pointing that out. I guess that means I should have set "MEDIAN_MAX_SIZE" to somewhat around 200 and then define a RunningMedian samples = RunningMedian(50); Well, as I learned yesterday the RunningMedian takes more processing time as it needs to sort the array. As I´m taking my measurements each 50ms I guess that would not be an option - especially not with a large array size (which seem absolutley logical now to limit the array size). I´m now using the RunningAverage-library as I found my optimum array size for my setup via the tests I did. So everything is fine now. But I still wonder if the array size can be made user configurable during runtime.

@ivanseidel: Thank you for pointing me to your library. But I allready implemented the RunningAverage-library by Rob Tillaart and only needed some fine tuning (and understanding on whats going on behind the scenes). The last results I got were suficiently constant. I just needed to adjust the array size to get the timing right (because a large array size means smoother data but more timeleg on steep gradients ).

Best regards,
Jan

I have tried to reproduce the getMedian() error but I could not. Using the [partial] data from your zip file I got no errors.

A possible cause I cannot verify is how much free RAM you have as it might be a problem during allocating the internal arrays as the lib uses dynamic memory. That could explain the behaviour seen.

But I still wonder if the array size can be made user configurable during runtime.

The lib could be rewritten to use dynamic memory and you could reallocate the memory. However as an Arduino has very limited RAM this gives fragmented (useless) memory quite fast.

that said, you might try with largest allocation first and gradually decrease the size to an optimum. Then the reallocation should less fragment the memory in theory ...