RunningAverage Library with sleepmode?

Hi, I am hoping to use this great RunningAverage library with a data logging Arduino project: Arduino Playground - RunningAverage.

I have the code working fine but when I introduce some SLEEP code into the routine to save power between samples the sketch starts producing corrupt output for the RunningAverage value (not the 'raw' value).... but only after it reaches the number of samples specified in RunningAverage valueRA(10); - before the 10th sample in this case, RunningAverage output is correct.

Wondering if anyone else has RunningAverage working OK with SLEEP code or can shed some light on what might be going wrong?

Doesn't look like RunningAverage should be bothered by sleep.

Do you do any calculations with millis()? With a deep enough sleep the timer will stop and produce unexpected results.

Hi John,

The code doesn't use millis, rather an RTC to produce an interupt to wake the processor. The board used is a WaspMote and the test code looks like this:

#include <WaspUtils.h>
#include "RunningAverage.h"
RunningAverage pressureRA(10);

const char* filename="data.csv";
float value_pressure = 0;
float avg_pressure = 0;
char aux_str[16];
int sample = 1;

void setup() 
{ 
   SensorAgr.setBoardMode(SENS_ON);  // Set Agriculture board ON
   SensorAgr.setSensorMode(SENS_ON, SENS_AGR_PRESSURE);
   SD.ON();
   RTC.ON();
   USB.begin();
   pressureRA.clr(); // explicitly start clean
   
  while (!SD.isSD())
  {Utils.blinkLEDs(200);
  delay(200);} 
  
  if(SD.create(filename)) 
  {Utils.blinkLEDs(200);
  delay(200);}  
} 

void loop()
{ 
    SensorAgr.sleepAgr("00:00:00:10", RTC_OFFSET, RTC_ALM1_MODE1, UART0_OFF | UART1_OFF | BAT_OFF, SENS_AGR_PLUVIOMETER);
    SensorAgr.setBoardMode(SENS_ON); // It is necessary to re-initiate the power supply 
    SensorAgr.setSensorMode(SENS_ON, SENS_AGR_PRESSURE);
    USB.begin(); // It is necessary to re-initiate the serial monitoring
    RTC.begin();
    SD.begin();
  
    //Sample the Pressure Sensor 
    value_pressure = SensorAgr.readValue(SENS_AGR_PRESSURE);
    value_pressure = (value_pressure * 10.0155);
    pressureRA.add(value_pressure);
    avg_pressure = pressureRA.avg();
    
    // Log the Pressure Sensor
    Utils.float2String(sample,aux_str, 0);
    SD.append(filename, aux_str);
    SD.append(filename,",");
    Utils.float2String(value_pressure,aux_str, 2);
    SD.append(filename, aux_str);
    SD.append(filename,",");
    Utils.float2String(avg_pressure,aux_str, 2);
    SD.append(filename, aux_str);
    SD.appendln(filename,",");
    USB.print("Sample: ");
    USB.println(sample);
    USB.print("Pressure RAW: ");
    USB.println(value_pressure);
    USB.print("Pressure AVG: ");
    USB.println(avg_pressure);
    
    sample ++;  
}
  • The RA lib uses malloc(), you might change the lib to use a hard coded array in the constructor and remove the free() in the destructor.
    (malloc should not interfere with sleep mode but you never know...)

Can you tell more about the value of the samples that you put into the class? Please Add this line ...

value_pressure = SensorAgr.readValue(SENS_AGR_PRESSURE);
Serial.println(value_pressure);
value_pressure = (value_pressure * 10.0155);

If the sensor reads all zero's the RA class would also return all zero's quite fast ...

Hi Rob,

Thanks for your input. I will try to change the array type... any chance of some more detailed intructions on what to do?

The average value fails instantly and the output to SD is also corrupt so susspect it could be the above.

Serial output:

Sample: 1
Pressure RAW: 1007.3206787109
Pressure AVG: 1007.3206787109
Sample: 2
Pressure RAW: 1012.8833618164
Pressure AVG: 1010.1020507812
Sample: 3
Pressure RAW: 1010.6582641601
Pressure AVG: 1010.2874145507
Sample: 4
Pressure RAW: 1006.2082519531
Pressure AVG: 1009.2676391601
Sample: 5
Pressure RAW: 1007.3206787109
Pressure AVG: 1008.8782348632
Sample: 6
Pressure RAW: 1007.3206787109
Pressure AVG: 1008.6186523437
Sample: 7
Pressure RAW: 1009.5458374023
Pressure AVG: 1008.7510986328
Sample: 8
Pressure RAW: 1015.1084594726
Pressure AVG: 1009.5457763671
Sample: 9
Pressure RAW: 1008.4332275390
Pressure AVG: 1009.4221801757
Sample: 10
Pressure RAW: 1007.3206787109
Pressure AVG: 1009.2120361328
Sample: 11
Pressure RAW: 1008.4332275390
Pressure AVG: 0.0000000000
Sample: 12
Pressure RAW: 1007.3206787109
Pressure AVG: 0.0000000000
Sample: 13
Pressure RAW: 1009.5458374023
Pressure AVG: 0.0000000000

SD card output:

1   1007.32   1007.32         
2   1012.88   1010.1         
3   1010.65   1010.28         
4   1006.2   1009.26         
5   1007.32   1008.87         
6   1007.32   1008.61         
7   1009.54   1008.75         
8   1015.1   1009.54         
9   1008.43   1009.42         
10   1007.32   1009.21         
11   1008.43   ./   )   (-*   (.00
12   1007.32   ./   )   (-*   (.00
13   1009.54   ./   )   (-*   (.00

Looks like Utils.float2String() is going wonky at the same time the average is coming out zero. Looks like it might be a memory overwrite problem of some kind. Nothing I can see in the RunningAverage library or in your code so I would suspect the WaspUtils library.

Thanks John, what you say makes sense...

How's your Spanish? (mine doesn't work :-).... wondering if you are able to spot any obvious issues in the float2String() function below:

00704                                                          {
00705 
00706         boolean neg = false;
00707  
00708         if( fl<0 ){
00709                 neg = true;
00710                 fl*=-1;
00711         }
00712  
00713         float numeroFloat=fl; //variable local e la que guardo el float para no machacarlo
00714  //defino una cadena de enteros para guardar la parte entera en sentido ascendente. Es decir el indice 0
00715         int parteEntera[10];
00716         int cifra; //guardo temporalmente la cifra
00717         long numero=(long)numeroFloat;  
00718         int size=0; //size nos da el numero de unidades del float: 1->decenas, 2->centenas etc. Asi sabemos el numero de cifras de la parte entera.
00719   
00720         while(1){
00721                 size=size+1;
00722                 cifra=numero%10;
00723                 numero=numero/10;
00724                 parteEntera[size-1]=cifra; //metemos en la cadena la cifra correspondiente
00725                 if (numero==0){
00726                         break;
00727                 }
00728         }//while
00729 
00730  //procesamos la parte entera del FLOAT
00731         int indice=0;
00732         if( neg ){
00733                 indice++;
00734                 str[0]='-';
00735         }
00736         for (int i=size-1; i>=0; i--)
00737         {          str[indice]=parteEntera[i]+'0'; //introduzco la cifra en ASCII dentro de la cadena str          
00738         indice++;
00739         }
00740 
00741  //ahora introduzco la 'coma' o 'punto' dentro del String
00742         str[indice]=',';
00743         indice++;
00744 
00745  //ahora procesamos la parte decimal del FLOAT
00746         numeroFloat=(numeroFloat-(int)numeroFloat);
00747         for (int i=1; i<=N ; i++)
00748         {
00749                 numeroFloat=numeroFloat*10;
00750                 cifra= (long)numeroFloat;          
00751                 numeroFloat=numeroFloat-cifra;
00752                 str[indice]=char(cifra)+48; //introduzco la cifra en ASCII dentro de la cadena str        
00753                 indice++;
00754         }    
00755 }

The partial source you provided uses a ',' for the decimal point so it's apparently not the same version you actually used.

Sorry John, that code was from the documentation.... below is what is actually getting compiled:

void WaspUtils::float2String (float fl, char str[], int N) {

	boolean neg = false;
 
	if( fl<0 ){
		neg = true;
		fl*=-1;
	}
 
	float numeroFloat=fl; 
	int parteEntera[10];
	int cifra; 
	long numero=(long)numeroFloat;  
	int size=0;
  
	while(1){
		size=size+1;
		cifra=numero%10;
		numero=numero/10;
		parteEntera[size-1]=cifra; 
		if (numero==0){
			break;
		}
	}

	int indice=0;
	if( neg ){
		indice++;
		str[0]='-';
	}
	for (int i=size-1; i>=0; i--)
	{
		str[indice]=parteEntera[i]+'0'; 
		indice++;
	}

	str[indice]='.';
	indice++;

	numeroFloat=(numeroFloat-(int)numeroFloat);
	for (int i=1; i<=N ; i++)
	{
		numeroFloat=numeroFloat*10;
		cifra= (long)numeroFloat;          
		numeroFloat=numeroFloat-cifra;
		str[indice]=char(cifra)+48;
		indice++;
	}
	str[indice]='\0';
}

I'm wondering if possibly re-initializing everything every time through loop() is causing a memory leak:

    USB.begin(); // It is necessary to re-initiate the serial monitoring
    RTC.begin();
    SD.begin();

Hi, The re-initializing is required after a SLEEP to start those services again - I tried omitting them but no joy. I'm not sure what I have done but the error is now slightly different - the averaged value is now slowly writing itself down to zero after the 10th sample (correctly written to SD also :slight_smile: - see below.

robtillaart mentioned removing the malloc but I'm not sure what this does or how to do it?

The RA lib uses malloc(), you might change the lib to use a hard coded array in the constructor and remove the free() in the destructor.
(malloc should not interfere with sleep mode but you never know...)

Sample: 8
Pressure RAW: 999.5330200195
Pressure AVG: 999.2548828125
Sample: 9
Pressure RAW: 997.3078002929
Pressure AVG: 999.0385131835
Sample: 10
Pressure RAW: 998.4204101562
Pressure AVG: 998.9767456054
Sample: 11
Pressure RAW: 999.5330200195
Pressure AVG: 908.1607055664
Sample: 12
Pressure RAW: 997.3078002929
Pressure AVG: 832.3879394531
Sample: 13
Pressure RAW: 997.3078002929
Pressure AVG: 768.3580932617
Sample: 14
Pressure RAW: 999.5330200195
Pressure AVG: 713.0780639648
Sample: 15
Pressure RAW: 998.4204101562
Pressure AVG: 665.6136474609
Sample: 16
Pressure RAW: 998.4204101562
Pressure AVG: 623.9432373046
Sample: 17
Pressure RAW: 998.4204101562
Pressure AVG: 587.3061523437
Sample: 18
Pressure RAW: 999.5330200195
Pressure AVG: 554.7399291992
Sample: 19
Pressure RAW: 999.5330200195
Pressure AVG: 525.6016845703
Sample: 20
Pressure RAW: 998.4204101562
Pressure AVG: 499.3215942382
Sample: 21
Pressure RAW: 997.3078002929
Pressure AVG: 475.438385009

The spanish float2string function looks quite similar to the standard Arduino print for floats. See no real problems with that code.

Looking at the samples ...

Sample: 19
Pressure RAW: 999.5330200195
Pressure AVG: 525.6016845703
Sample: 20
Pressure RAW: 998.4204101562
Pressure AVG: 499.3215942382

What is remarkable is that the average drops, with approx 25 per step although the raw values give no reason for that. They are all somewhere around 1000. That could means that the internal values are overwritten somehow. Strange....

=> add code to print the internals in RA.add()

void RunningAverage::add(float f)
{
Serial.println(f,2);
Serial.println(_sum,2);
	_sum -= _ar[_idx];
	_ar[_idx] = f;
	_sum += _ar[_idx];
Serial.println(_sum,2);
	_idx = (_idx + 1) % _size;
	if (_cnt < _size) _cnt++;
Serial.println(_idx);
Serial.println(_cnt);
}

This would show how the internal variables change..

Hi Rob,

WaspMote uses USB.print rather than Serial.print which is making it difficult for me to print (f) & (_sum) to serial.... (Waspmote doesn't recognise Serial.begin etc and USB.print in the cpp files causes a compile error despite #include <WaspUtils.h>) is there another way I can expose these variables... ?

I'm wondering if I should give your function below a whirl to see if this improves things? Not sure how to use it for floats though?

long runningAverage(int M)
{
  static int LM[10];      // LastMeasurements
  static byte index = 0;
  static long sum = 0;
  static byte count = 0;

  // keep sum updated to improve speed.
  sum -= LM[index];
  LM[index] = M;
  sum += LM[index];
  index = index % LMSIZE;
  if (count < LMSIZE) count++;

  return sum / count;
}

You could do that but be aware the function you posted is just an integer variant of the Running AVerage class.

Hi Rob,

How would I update this function to use floats please? This will give me a chance to debug :slight_smile:

Check - Arduino Playground - RunningAverage -

the floating point version of the library is at the bottom . Just replace the content of the files (optionally replace the malloc as suggested earlier)
.

Some theories:

Theory 1: the internal array is overwitten somehow.
Test: make a running average with different sizes e.g. 20, 15, 5 and 1 to see if the problem depends on the size of the internal array. Do you see patterns arise?

Theory 2: the values in RAM are lost during sleep as they don't get refreshed or so?
Test: dump the internals of the class before entering and recovering from sleepmode

Theory 3: some internal values of the RA class are optimized in registers that are cleared during sleepMode
Test: Make all vars from the RA class volatile to test this theory..

TIA for testing,
Rob

Hi all.

As is the case in this post I'm also working with a Waspmote. I like to include my own libraries to the compiler, as you did with that RunningAverage. I fully understand the code to be included in each file in both. H as in the. Cpp. The question is ... what steps I have to continue to include these two files in the IDE Waspmote? That is, within the IDE Waspmote, what are the directories that I have to enter and leave my files to compile when not of any problems and can have this code apart and accessible via the include command.

Thank you very much, it would be helpful

See the Notes section on - Arduino Playground - RunningAverage -

(not familiar with IDE waspmote and if it has other requirements than the Arduino IDE)