Weird values store... Sometimes

Hi there,
Still traveling within the first batches of tests with iot, through my wifi 1010 mkr measuring some sensors values.

I noticed another thing weird, the first thing being a question still not solved in another thread where some integer variables gives some float in the arduino cloud widgets... Which is VERY weird.

The new thing is
I'm measuring temperature with a timer in the arduino. Each 1s, it reads the sensor, stores it in the last index position of an array with a size of 60,and do a mean. This current running mean is stored in a float variable, the one updating the cloud.

It is a cheap and easy process to smooth. Working fine in all other cases (of my programmer's life).

But here, it gives me this kind of thing ::

Some values, only one sometimes at a time, usually, are like that one.
I could imagine this is a sensing mistake... But come one... Smoothed with a running mean over 60 measures?!

I suspect something else.

Did you already get this?

Maybe the sensor made a faulty reading?
Seems unlikely........

As I described,

I read one time each 1s, do a running mean (mean done on last 60 readings) and update it on the cloud.

If it does even 1 or 2 or 3 bad readings, it woild be smoothed and or we'd see the weird thing not only on just one value but more.

I suspect the cloud update.
As the one I noticed in another previous post.

Me too.
I am guessing that the data could have been corrupted.
During the transmission.
Sadly I have no idea how to fix that.
Maybe by passing the data back to the arduino and checking it?
You could use some variables and some if’s to do that.

1 Like

If I can't trust the arduino iot cloud (which, I think, seemed to be very reliable), that wouldnt be OK.

This is why I suppose I did something bad.
I can provide the code.

Which other services could we use for logging data and which also provide an easy way to get them from a webclient?

I currently have no ideas about what is wrong.
Still thinking.
I give you a list of cloud providers soon(1-2 days?).
@structurevoid
what do you mean with this?

Sorry haven't had the time to look for cloud services.

1 Like

It is quite impossible that data gets corrupt during trasmission to the cloud :slight_smile:
You need to investigate more about the issue in your algorithm. I recommend you post your code here so that multiple eyes can help spot the issue

1 Like

This is the first thing I thought & I still think.
I probably messed something.

Here it is, not a secret nor science rocket :slight_smile:

#include "thingProperties.h"
#include <math.h>

#if defined(ARDUINO_ARCH_AVR)
#define debug  Serial
#elif defined(ARDUINO_ARCH_SAMD) ||  defined(ARDUINO_ARCH_SAM)
#define debug  SerialUSB
#else
#define debug  Serial
#endif


// GROVE TEMPERATURE
const int B = 4275;               // B value of the thermistor
const int R0 = 100000;            // R0 = 100k
const int pinTempSensor = A0;     // Grove - Temperature Sensor connect to A0
float currentTemperature;


// inner timerS
unsigned long previousMillis = 0;
const long interval = 1000; //milliseconds


const int numReadings = 60;
float readings[numReadings];  // the readings from the analog input
int readIndex = 0;            // the index of the current reading
float total = 0;              // the running total
float average = 0;            // the average


void setup() {

  Serial.begin(9600);
  delay(1500);


  // initialize all the readings to 0
  for (int thisReading = 0; thisReading < numReadings; thisReading++) {
    readings[thisReading] = 0;
  }

  initProperties();
  ArduinoCloud.begin(ArduinoIoTPreferredConnection);

  setDebugMessageLevel(2);
  ArduinoCloud.printDebugInfo();
}

void loop() {
  ArduinoCloud.update();

  unsigned long currentMillis = millis();

  if (currentMillis - previousMillis >= interval) {
    previousMillis = currentMillis;

    // TEMPERATURE
    int a = analogRead(pinTempSensor);
    float R = 1023.0 / a - 1.0;
    R = R0 * R;
    currentTemperature = 1.0 / (log(R / R0) / B + 1 / 298.15) - 273.15; // convert to temperature via datasheet

    // subtract the last reading:
    total = total - readings[readIndex];
    // read from the sensor:
    readings[readIndex] = currentTemperature;
    // add the reading to the total:
    total = total + readings[readIndex];
    // advance to the next position in the array:
    readIndex = readIndex + 1;

    if (readIndex >= numReadings) {
      readIndex = 0;
    }

    // calculate the average:
    temperature = total / numReadings;
  }
}

The thing still happening :


I was suspecting something about unsynced timer.
My measure in the loop VS the cloud update. I asked something in another post about this.
But I can't imagine this.
Things happening sequentially here.

So the loop : cloud update, measure, mean calculation, variable updated.

Etc etc.

Couldn't be that.
But if I can't have more informations about this, I should find another cloud/system for sure.

Actually, I'd like to keep it but to store values in a Google sheets too. Just for comparing. Maybe.

Have you tried to log a variable with a simpler logic? For example i have an uptime counter incrementing each second and i'm logging the values in a chart like you do. The line in smooth without any artifact or spike.

image
image

1 Like

doing this right now. thanks for this idea :slight_smile:

by each second, you mean update time in the cloud ? or another timer ?

The sketch is very basic :slight_smile:

void loop() {
  ArduinoCloud.update();
  // Your code here 
  potentiometer = analogRead(A0);
  seconds = millis() / 1000;
}

1 Like

did that with a CloudCounter "type" variable.

now counting :slight_smile:

thanks @pennam

Hi there,

zoomed with an interval about to less than 1 min, it gives this:

I'm not surprised.
There are 'like' two timers.

The moment when the integer goes from a value to the n+1 one (as I'm doing exactly millis()/1000) is not synced with the arduino updating "clock"

i suspect this is a render issue, can you check your widget settings? i'm using linear interpolation

image

line here too.

I suspect something else....

At first, I fixed something in my code.
The mean was always considering that 60 values were already measured. It was missing one case: the first 60 measures.
Ok.

Indeed, I was suspecting some reboots.
And in that case, the first 59 measures would have been unconsistent.

here is the new one for the record, and those interested:

// STRUCTURE VOID

/*

  temperature
  - 1 samp pers second
  - running mean calculation considering the latest 60 measures
  - handling the case of the first 59 measures

*/



#include "thingProperties.h"
#include <math.h>

#if defined(ARDUINO_ARCH_AVR)
#define debug  Serial
#elif defined(ARDUINO_ARCH_SAMD) ||  defined(ARDUINO_ARCH_SAM)
#define debug  SerialUSB
#else
#define debug  Serial
#endif


// GROVE TEMPERATURE
const int B = 4275;               // B value of the thermistor
const int R0 = 100000;            // R0 = 100k
const int pinTempSensor = A0;     // Grove - Temperature Sensor connect to A0
float currentTemperature;

// inner timerS
unsigned long previousMillis = 0;
const long interval = 1000; //milliseconds

const int numReadings = 60;
int alreadyStored = 0;

float readings[numReadings];  // the readings from the analog input
int readIndex = 0;          // the index of the current reading
float total = 0;              // the running total
float average = 0;            // the average


void setup() {
  // Initialize serial and wait for port to open:
  Serial.begin(9600);
  // This delay gives the chance to wait for a Serial Monitor without blocking if none is found
  delay(1500);

  // initialize all the readings to 0:
  for (int thisReading = 0; thisReading < numReadings; thisReading++) {
    readings[thisReading] = 0;
  }

  // Defined in thingProperties.h
  initProperties();

  // Connect to Arduino IoT Cloud
  ArduinoCloud.begin(ArduinoIoTPreferredConnection);

  setDebugMessageLevel(2);
  ArduinoCloud.printDebugInfo();
}

void loop() {
  ArduinoCloud.update();

  unsigned long currentMillis = millis();
  upt = millis() / 1000;

  if (currentMillis - previousMillis >= interval) {
    previousMillis = currentMillis;

    // TEMPERATURE
    int a = analogRead(pinTempSensor);
    float R = 1023.0 / a - 1.0;
    R = R0 * R;
    currentTemperature = 1.0 / (log(R / R0) / B + 1 / 298.15) - 273.15; // convert to temperature via datasheet
    //Serial.println(temperature);

    // subtract the last reading:
    total = total - readings[readIndex];
    // read from the sensor:
    readings[readIndex] = currentTemperature;
    // add the reading to the total:
    total = total + readings[readIndex];
    // advance to the next position in the array:
    readIndex = readIndex + 1;
    
    

    // if we're at the end of the array...
    if (readIndex >= numReadings) {
      // ...wrap around to the beginning:
      readIndex = 0;
    }

    // calculate the average:
    if (alreadyStored< numReadings) alreadyStored++;
    
    temperature = total / alreadyStored;
  }
}

Ok but it doesn't explain the "only one value bad" issue.

So I suspect something.

If the hardware were rebooting, how the cloud update routine would behave ???

Here is what the widget gives:

I implemented the uptime control around 12:00
And I just implemented the handling of first 59 values. GOOD. no more problem.

Now waiting for the next "possible" reboot issue to see if there is a hole in the data, it would be something else.

1 Like

Things seem okay since yesterday update.
It has fixed the thing but I didn't identify the cause.

For the record.
Data were totally OK in the board, the adaptative recurring mean working fine. And even before the last update, it should have been ok.

The only thing that could have happened, on my opinion, was a kind of auto reboot/reset.
Thought about a kind of watchdog issue.
But the arduino cloud update reset the watchdog. And actually, I don't like the idea to disable it.

No electricity fault. And my two different boards with same code were running in... Two different places and issues were rye same.

More infos soon.