Solved: BME280 weather station: averaging and ring buffer

I'm sure this is a simple problem but I cant see how to do this.
I have a program which will need to calculate several averages of values; so - to me - rather than have lots of repeated code, it would make sense to make this a function.

So eg (forgive any erors in syntax)

float findaverage(number, count){
float value=0;
int i=0;
value+=number;
i++;
if(i == count) {value /=count;}
return(value);
}

I know it cant work as is; but is there something simple I'm missing?

float value=0;
int i=0;

Sets value and i to zero each time the function is called so

value+=number;

just sets value to number
incrementing i just makes i equal to 1 so why bother ?

If you want the function not to reset value and i to zero each time that it is called then declare them static

Please post an example of how you are trying to use the function

I have a program which will need to calculate several averages of values;

You'd need a running total for each set of values, so a simple class would seem to me to fit the bill.

TheMemberFormerlyKnownAsAWOL:
You'd need a running total for each set of values, so a simple class would seem to me to fit the bill.

See runningAve in LC_baseTools. ( IDE lib manager ) Stuff in a number, pops out an average of the last N numbers. Make as many as you like.

-jim lee

float
findaverage (
    float   vals [],
    int     nVals )
{
    float sum = 0;

    for (int n = 0; n < nVals; n++)
        sum += vals [n];

    return sum / nVals;
}

#include <stdio.h>

int
main ()
{
    float  data [] = {1.2, 3.3, 4.2, 2.2};
    printf (" %.2f\n", findaverage (data, 4));
    return 0;
}

Thanks for your replies;
I didnt really want to use an array as it means storing all the values.
If I put the averaging in the main program I can just initialise, keep a count and a running total and work out the average when all "count" values have been recorded.

@TheMemberFormerlyKnownAsAWOL:

You'd need a running total for each set of values, so a simple class would seem to me to fit the bill.

@UKHeliBob:

If you want the function not to reset value and i to zero each time that it is called then declare them static

From the (VERY) little I know of OOP it would seem that this task is almost a classical case for using it.
I'll need to think further about this.

then keep update a running average

float
findaverage (
    float   val )
{
    static float avg = 0;

    if (0 == avg)
        return avg = val;

    return avg += (val - avg) / 8;
}

#include <stdio.h>

int
main ()
{
    float  data [] = {1.2, 3.3, 4.2, 2.2};

    for (int n = 0; n < 4; n++)
        printf (" %.2f\n", findaverage (data [n]));
    return 0;
}

How many values are there to average?

This function here, as an example, takes a running average of 10 values, and only casts to a float when returning the result. Not only do you get a real-time rolling average (don't need to wait for all the values), you also get an order of magnitude better precision. For example, if reading ADC, the resolution won't be ±1 count, it'll be ±0.1 count. Can you spare 20-ish bytes?

float movingAVG(int value) {
  static int arrayDat[10];
  static byte pos;
  static long sum;
  pos++;
  if (pos >= 10) pos = 0;
  sum = sum - arrayDat[pos] + value;
  arrayDat[pos] = value;
  return (float)sum / 10.0;
}

void setup() {
  Serial.begin(9600);
  // get the average of only the 10 most recent values
  // run this 11 times (11 values entered)
  // show one significant digit to the right of decimal
  for (int i = 0; i <= 10; i++) {
    Serial.println((movingAVG(1)), 1);
  }
}

void loop() {
}

EDIT: This is a moving (rolling) average. For more than 1 set of values, see reply#2.

gcjr:
then keep update a running average

float

findaverage (
   float   val )
{
   static float avg = 0;

if (0 == avg)
       return avg = val;

return avg += (val - avg) / 8;
}

#include <stdio.h>

int
main ()
{
   float  data [] = {1.2, 3.3, 4.2, 2.2};

for (int n = 0; n < 4; n++)
       printf (" %.2f\n", findaverage (data [n]));
   return 0;
}

Have you missed something? Using the values 0.0, 0.0, 0.0, 4.0 would give averages of 0.0, 0.0, 0.0, 4.0 rather than 1.0 for the last one? Using 2.0 and 4.0 would give 2.0 and 2.25 rather than 2.0 and 3.0?

gcjr:
then keep update a running average

float

findaverage (
    float  val )
{
    static float avg = 0;

if (0 == avg)
        return avg = val;

return avg += (val - avg) / 8;
}

#include <stdio.h>

int
main ()
{
    float  data [] = {1.2, 3.3, 4.2, 2.2};

for (int n = 0; n < 4; n++)
        printf (" %.2f\n", findaverage (data [n]));
    return 0;
}

That is not an average, it is a low-pass filter, with a filter coefficient of 0.125.

jimLee:
See runningAve in LC_baseTools. ( IDE lib manager ) Stuff in a number, pops out an average of the last N numbers. Make as many as you like.

-jim lee

LC_baseTools runningAvg has a couple of bugs - one in getMin, and one in getMax - the for loops should be for (int i = 1; i < numValues; i++)

(Not relevant to this discussion, but they ought to be fixed. And the for loop in addData could be eliminated completely by the addition of one more class variable.)

countrypaul:
Have you missed something? Using the values 0.0, 0.0, 0.0, 4.0 would give averages of 0.0, 0.0, 0.0, 4.0 rather than 1.0 for the last one? Using 2.0 and 4.0 would give 2.0 and 2.25 rather than 2.0 and 3.0?

i provided code for an exact (mean) average over a specific # of samples. this is a running avg that can be used to estimate an avg for each of hundreds of samples.

it's not clear what the OP is asking. presumably my first suggested wasn't what he wanted

Perhaps smoothing is the desired result?

I use the SimpleKalmanFilter to smooth data.

Using a default value of .01 to set the Process Variance works but using a milli or micro count measurement functions would be best.

Here is an example of using the SimplaeKalmanFilter with updating the Process Variance. Updating the Process Variance before calculations improves smoothing.

#include <SimpleKalmanFilter.h> is added in the global declaration section.

void fReadAD( void * parameter )
{
  float    ADbits = 4095.0f;
  float    uPvolts = 3.3f;
  float    adcValue_b = 0.0f; //plant in yellow pot
  uint64_t TimePastKalman  = esp_timer_get_time(); // used by the Kalman filter UpdateProcessNoise, time since last kalman calculation
  float    WetValue = 1.07f; // value found by putting sensor in water
  float    DryValue = 2.732f; // value of probe when held in air
  float    Range = DryValue - WetValue;
  float    RemainingMoisture = 100.0f;
  SimpleKalmanFilter KF_ADC_b( 1.0f, 1.0f, .01f );
  for (;;)
  {
    xEventGroupWaitBits (eg, evtADCreading, pdTRUE, pdTRUE, portMAX_DELAY ); //
    adcValue_b = float( adc1_get_raw(ADC1_CHANNEL_3) ); //take a raw ADC reading
    adcValue_b = ( adcValue_b * uPvolts ) / ADbits; //calculate voltage
    KF_ADC_b.setProcessNoise( (esp_timer_get_time() - TimePastKalman) / 1000000.0f ); //get time, in microsecods, since last readings
    adcValue_b = KF_ADC_b.updateEstimate( adcValue_b ); // apply simple Kalman filter
    TimePastKalman = esp_timer_get_time(); // time of update complete
    RemainingMoisture = 100.0f * (1 - ((adcValue_b - WetValue) / (DryValue - WetValue))); //remaining moisture =  1-(xTarget - xMin) / (xMax - xMin) as a percentage of the sensor wet dry volatges
    xQueueOverwrite( xQ_RM, (void *) &RemainingMoisture );
    log_i( "adcValue_b = %f remaining moisture %f%", adcValue_b, RemainingMoisture );
  }
  vTaskDelete( NULL );
}

TheMemberFormerlyKnownAsAWOL:
LC_baseTools runningAvg has a couple of bugs - one in getMin, and one in getMax - the for loops should be

for (int i = 1; i < numValues; i++)

(Not relevant to this discussion, but they ought to be fixed. And the for loop in addData could be eliminated completely by the addition of one more class variable.)

Ye gads! You are correct!

Sad day..

Anyhow, the min/max bug is fixed and a new version of LC_baseTools has been released.

P.S. I left in the loop. The idea behind the loop was to keep any long term error from building up. Everything is in floats so I figured that adding and subtracting them could get kinda' "fuzzy" and drift after awhile.

-jim lee

I'm overwhelmed with all these responses - sorry not to get back sooner, its taken me a while to review them all.

@gcjr: Thanks, I've seen from your code how a static declaration works; however on running the code I get these results: 1.20, 1.46, 1.80, 1.85 - which are not a running average from the data,

it is a low-pass filter, with a filter coefficient of 0.125.

@jimLee: I'll give that a try. Your "runningAvg.ino" example is a big help in understanding how to use it.
Do I just put "runningAvg.cpp" and "runningAvg.h" in the same folder as the sketch?

To clarify what I am intending I have built the "simple ESP8266 weather station"

The pressure readings are taken very frequently so the web page can update.

I'm extending it to provide an indication of the trend of the pressure reading
so the plan is to:

1: average the readings over about 1 second to reduce noise, and show on the display;

I then want to get an indication of the change over say a 6 hour period. So to reduce the data I plan to

2: average the result from above over say 5 minutes.

3: The results will be fed to a ring buffer so I can compare on a 5 minute basis the current pressure (averaged over 5 min) with that from 6 hours ago.

countrypaul:
Have you missed something? Using the values 0.0, 0.0, 0.0, 4.0 would give averages of 0.0, 0.0, 0.0, 4.0 rather than 1.0 for the last one? Using 2.0 and 4.0 would give 2.0 and 2.25 rather than 2.0 and 3.0?

The logic is wrong though, as it treats runs of initial zeroes as if they are not there.

The code should be:

float running_average (float   val)  // give a non-misleading name!
{
    static float avg;
    static bool first_sample = true ;

    if (first_sample)
    {
        first_sample = false ;
        return avg = val;
    }

    return avg += (val - avg) / 8;
}

As you want 0,0,0,4,4,... to yield 0,0,0,0.5,0.9375,....

@jimLee: I'll give that a try. Your "runningAvg.ino" example is a big help in understanding how to use it.
Do I just put "runningAvg.cpp" and "runningAvg.h" in the same folder as the sketch?

No, its even easier.

If you install LC_baseTools using the library manager (in the IDE). You can compile and run it just as it stands. Anything in that library will be available to you by just adding in the correct

#include <blabla.h>

for it to your source files.

-jim lee

Looking at the advanced info and data sheet for the BME280 I realised I did not need to do the noise cancellation in code, as it can be done by the BME280 just by changing the settings.
So my averaging and ring buffer only needed to occur once.

Its all very basic, but I've put the code here. Changed the topic title to better reflect the content.

Thanks all for your help

/*
   Code from https://lastminuteengineers.com/bme280-esp8266-weather-station/
   AJAX code from above page added in.

   Modified to:
   change connection layout to match the BME280 module to the NodeMCU pins;
   change the settings on the BME280 to provide better stability of all readings;
   added code to average readings over a 5 minute interval
   added code for a ring buffer to allow comparison of current and "old" reading from 3 hours ago.
   change the html layout to allow for the new entries
*/

#include <ESP8266WebServer.h> //https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266WebServer
#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BME280.h>

// order connections to match bme module layout
#define BME_MISO D1
#define BME_CS D2
#define BME_MOSI D3
#define BME_SCK D4

//Adafruit_BME280 bme; //I2C
Adafruit_BME280 bme(BME_CS, BME_MOSI, BME_MISO, BME_SCK); // software SPI

float temperature, humidity, pressure, change; //indicate pressure +rising or -falling

//calculate 5 min average
int  t_interval = 5000;
float p_ave = 0;
int count5 = 60; //60 * 5 sec in 5 min assuming t_interval = 5000msec

//ring buffer to allow trend detection
const int length=36; //size of buffer: 36 places allows a time interval of 36 * 5 min = 3 hours
float data[length]; //buffer array
int put_index=0; //index to array
float p_now, p_was;//current pressure and old pressure

/*Put your SSID & Password*/
const char* ssid = "TALKTALKCC34C4";  // Enter SSID here
const char* password = "MVGGX799";  //Enter Password here

ESP8266WebServer server(80);

void setup() {
  Serial.begin(115200);

  //initialise array values to zero
  for(put_index=0; put_index<length; put_index++){
    data[put_index]=0;
  }
  put_index=0; //reset ready for data entry
  change = 0;
  delay(100);

  bme.begin(0x76);
  //code from BME280 advanced settings example & modified for more stable readings at the expense of more power consumption
  bme.setSampling(Adafruit_BME280::MODE_NORMAL,
                  Adafruit_BME280::SAMPLING_X16,  // temperature
                  Adafruit_BME280::SAMPLING_X16, // pressure
                  Adafruit_BME280::SAMPLING_X16,  // humidity
                  Adafruit_BME280::FILTER_X16,
                  //Adafruit_BME280::STANDBY_MS_0_5 );
                  Adafruit_BME280::STANDBY_MS_250 ); //standby to 250msec
  // 1 + (2 * T_ovs) + (2 * P_ovs + 0.5) + (2 * H_ovs + 0.5)
  // T_ovs = 16 ; // P_ovs = 16 ; // H_ovs = 16 ;    // = 50ms
  // with standby time that should really be 300ms - take no more than 1 reading per second


  Serial.println("Connecting to ");
  Serial.println(ssid);

  //connect to your local wi-fi network
  WiFi.begin(ssid, password);

  //check wi-fi is connected to wi-fi network
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected..!");
  Serial.print("Got IP: ");  Serial.println(WiFi.localIP());

  server.on("/", handle_OnConnect);
  server.onNotFound(handle_NotFound);

  server.begin();
  Serial.println("HTTP server started");

}
void loop() {

  for (int i = 0; i < count5; i++) {
    temperature = bme.readTemperature();
    humidity = bme.readHumidity();
    pressure = bme.readPressure() / 100.0F; //bme280
    server.handleClient();
    delay(t_interval); 
    p_ave += pressure;
  }
  p_ave /= count5;
  p_was = put(p_ave);
  change = p_ave - p_was; //change reading
  p_ave = 0; //reset ready for next calculation
}

float put(float value){
  Serial.print("p-now = ");
  Serial.print(value, 2);
  data[put_index] = value;
  put_index=(put_index+1)% length;
  value=data[put_index];  
  Serial.print(":  p-was = ");
  Serial.println(value, 2);
  return(value);
}


void handle_OnConnect() {
   server.send(200, "text/html", SendHTML(temperature, humidity, pressure, change));
}

void handle_NotFound() {
  server.send(404, "text/plain", "Not found");
}

String SendHTML(float temperature, float humidity, float pressure, float change) {
  String ptr = "<!DOCTYPE html> <html>\n";
  ptr += "<head><meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0, user-scalable=no\">\n";
  ptr += "<title>ESP8266 Weather Station</title>\n";
  ptr += "<style>html { font-family: Helvetica; display: inline-block; margin: 0px auto; text-align: center;}\n";
  ptr += "body{margin-top: 50px;} h1 {color: #444444;margin: 50px auto 30px;}\n";
  ptr += "p {font-size: 24px;color: #444444;margin-bottom: 10px;}\n";
  ptr += "</style>\n";
  //AJAX code to do the refresh
  ptr += "<script>\n";
  ptr += "setInterval(loadDoc,1000);\n";
  ptr += "function loadDoc() {\n";
  ptr += "var xhttp = new XMLHttpRequest();\n";
  ptr += "xhttp.onreadystatechange = function() {\n";
  ptr += "if (this.readyState == 4 && this.status == 200) {\n";
  ptr += "document.body.innerHTML =this.responseText}\n";
  ptr += "};\n";
  ptr += "xhttp.open(\"GET\", \"/\", true);\n";
  ptr += "xhttp.send();\n";
  ptr += "}\n";
  ptr += "</script>\n";
  ptr += "</head>\n";
  ptr += "<body>\n";
  ptr += "<div id=\"webpage\">\n";
  ptr += "<h1>ESP8266 Weather Station</h1>\n";
  ptr += "<p>Temperature: ";
  ptr += temperature;
  ptr += "&deg;C</p>";
  ptr += "<p>Humidity: ";
  ptr += humidity;
  ptr += "%</p>";
  ptr += "<p>---</p>";
  ptr += "<p>950:stormy; 970: rain; 1000: changeable; 1030: fair; 1060: very dry </p>";
  ptr += "<p>Pressure: ";
  ptr += pressure;
  ptr += "hPa</p>";
   ptr += "<p>---</p>";
  ptr += "<p>-6: rapid fall; -3: fall; -1 to +1 steady; +3 rising; +6 rapid rise;</p>";
  ptr += "<p>3 Hour Change: ";
  ptr += change;
  ptr += "hPa</p>";
  ptr += "</div>\n";
  ptr += "</body>\n";
  ptr += "</html>\n";
  return ptr;
}

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.