Standard deviation calculation generating massive -incorrect- numbers

Hello!

Long time seeker of knowledge from the forums, the first time I have had to reach out for help. I am trying to work out the standard deviation of an array s_val and I am following this blog post as upon finding it, I thought I could follow the code, but I am having strange results.

I know I should be posting my whole code, and I will at the bottom of the post but im fairly confident im just not declaring my values as the correct type of numbers, here's the bit im falling over on:

//variables:
float s_val[4] = {1100,1100,1100,1100};
float meanSample;
float sqDevSum = 0.0;
int SAMPLES = 4; 

//there is some more code here//

 for(int i = 0; i < SAMPLES; i++) {
    // pow(x, 2) is x squared.
    sqDevSum += pow((meanSample - float(s_val[i])), 2);
    Serial.print("s_val - mean squared - ");
    Serial.println(sqDevSum);
  }

you see I have been printing out the values for sqDevSum as that's where the system loses the plot, I am expecting very low values for sqDevSum, the data set is very condensed and the values all very similar while I write the programme, (I plan to ultimately sample data once an hour) so lets say s_val[0] is 1015.23 and the meanSample is 1017.76 (I have verified meansample is correct from an earlier point of the code) that's 1017.76 - 1015.23 = 2.53 and 2.532is 6.4009 so on the first go through the for loop, I would expect to see 6.4009 and three returns of 0 as the s_val array only has info in s_val[0] right at the start, however, this is the Serial readout i get:

13:16:52.334 -> 1028.21
13:16:52.709 -> END OF LOOP
13:16:52.709 -> s_val - mean squared - 594684.68
13:16:52.746 -> s_val - mean squared - 660760.75
13:16:52.746 -> s_val - mean squared - 726836.81
13:16:52.783 -> s_val - mean squared - 792912.87

that figure of 1028.21 is the data point that gets shifted into s_val[0]

Im in well over my head, but im pretty sure im just not handling the numbers as they go through the maths as the correct "type" float, int, etc . . . can anyone see why I get such high values in my readout? here's the whole code, it's in a state of being written, and there's a whole bunch of superfluous i2c stuff going on, but my hunch could well be wrong:

#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BME280.h>
#include <ds3231.h>

Adafruit_BME280 bme;

ts t; //ts is a struct findable in ds3231.h


float pressureVals[33];
float s_val[4] = {1100,1100,1100,1100};
float var;
float meanSample;
float stDev;
float multiplierCheck;
float pressureSum;
float avgPressure;
float sampleSum = 0;
float sqDevSum = 0.0;
int multiplier;
int SAMPLES = 4;

void setup() {
// -------RTC-------
  
  Wire.begin(); //start i2c (required for connection)
  DS3231_init(DS3231_INTCN); //register the ds3231 (DS3231_INTCN is the default address of ds3231, this is set by macro for no performance loss)

// --------BME-------- 

  bme.begin(0x76);

// -------Other-------

  Serial.begin(9600);

}
void loop() {
  pressureVals[0] = (bme.readPressure() / 100.0F);
  for (int i = 32; i >= 0; i--) {
  pressureVals[i+1] = pressureVals[i];
  Serial.print(pressureVals[i]);
  Serial.println();
  delay(350);
}
  
Serial.print("END OF LOOP");
Serial.println();

  float sampleSum = 0;
  s_val[0] = pressureVals[1];
  s_val[1] = pressureVals[3];
  s_val[2] = pressureVals[5];
  s_val[3] = pressureVals[9];

  for(int i = 0; i < SAMPLES; i++) {
    sampleSum += s_val[i];
  }
  
  meanSample = sampleSum/float(SAMPLES);

  for(int i = 0; i < SAMPLES; i++) {
    // pow(x, 2) is x squared.
    sqDevSum += pow((float(s_val[i]) - meanSample), 2);
    Serial.print("s_val - mean squared - ");
    Serial.println(sqDevSum);
  }

  stDev = sqrt(sqDevSum/float(SAMPLES));

Serial.print("sample sum - ");
Serial.println(sampleSum);
Serial.print("Mean of the Samples - ");
Serial.println(meanSample);
Serial.print("sum the squares of the differences from the mean - ");
Serial.println(sqDevSum);
Serial.print("hopefully, a standard deviation? - ");
Serial.println(stDev);
}

Always post ALL your code. In the useless snippet, the value of meanSample is undefined.

Don't use pow() for squares, it is slow and inaccurate. Use multiply instead.

    sqDevSum += (meanSample - s_val[i])*(meanSample - s_val[i]);

What is this line supposed to do? Does the posted code actually compile?

Serial.println(sampleSum);s_val

Otherwise, I don't see a problem. If you do think there is a problem with a given calculation, print out all the starting values, the intermediate values, the final values, and use your calculator to check the results.

@jameath

Other post/duplicate DELETED
Please do NOT cross post / duplicate as it wastes peoples time and efforts to have more than one post for a single topic.

Continued cross posting could result in a time out from the forum.

Could you also take a few moments to Learn How To Use The Forum.
It will help you get the best out of the forum in the future.
Other general help and troubleshooting advice can be found here.
It will help you get the best out of the forum in the future.

Try debug prints on more than just one variable.

@ballscrewbob

I didn't post this anywhere else? I tried to edit a few spelling mistakes and got an angry "your posting too quickly message" that might have re-posted the whole thing? either way, sorry, but I didn't do that.

@jremington

Thanks for the pointer on squares! as that is the line where it all falls over, ill simplify it straight away. it didn't compile any more! looks like I typed into the IDE by mistake, I have amended the post and my code . . . the downsides of having several screens I guess :stuck_out_tongue:

@aarg

that's what I have been doing to find where it all fell over, will post some more returns after making some tweaks, thanks!

jameath:
Thanks for the pointer on squares! as that is the line where it all falls over, ill simplify it straight away.

No, it is not the line where it falls over. He just wanted to clean it up.

aarg:
No, it is not the line where it falls over. He just wanted to clean it up.

Cheers, that really helps! thank you so much

You were missing a few steps. Calculating the mean. Taking the mean of the squares. Taking the square-root.

const int SAMPLES = 4;
//variables:
float s_val[SAMPLES] = {1100.0, 1100.0, 1100.0, 1100.0};
float meanSample;
float sqDevSum = 0.0;




//there is some more code here//
void setup()
{
  Serial.begin(115200);


  s_val[0] = 1028.21;
  Serial.print("New sample: ");
  Serial.println(s_val[0]);


  // Calculate meanSample
  float sum = 0.0;
  for (int i = 0; i < SAMPLES; i++)
  {
    sum += s_val[i];
  }
  meanSample = sum / SAMPLES;
  Serial.print("meanSample: ");
  Serial.println(meanSample);


  // Calculate standzrd deviation?
  for (int i = 0; i < SAMPLES; i++)
  {
    sqDevSum += (meanSample - s_val[i]) * (meanSample - s_val[i]);
  }
  Serial.print("Sum of the squares: ");
  Serial.println(sqDevSum);


  Serial.print("Standrd Deviation: ");
  Serial.println(pow(sqDevSum / SAMPLES, 0.5));  // Square-root of the mean of the squared deviations
}


void loop() {}

Results:

New sample: 1028.21
meanSample: 1082.05
Sum of the squares: 3865.36
Standard Deviation: 31.09

Hint: there is a mathematical trick for calculating the standard deviation that reduces code and computation:

const int SAMPLES = 4;
//variables:
float s_val[SAMPLES] = {1028.21, 1100.0, 1100.0, 1100.0};
float meanSample;
float sdDev = 0.0;

void setup()
{
  Serial.begin(115200);
  while(!Serial);

  // Calculate meanSample
  float sum = 0.0, s2sum=0;
  for (int i = 0; i < SAMPLES; i++)
  {
    sum += s_val[i];
    s2sum += s_val[i]*s_val[i];  //sum of squared input values
  }
  meanSample = sum / SAMPLES;
  Serial.print("meanSample: ");
  Serial.println(meanSample);

  sdDev = sqrt(s2sum/SAMPLES-meanSample*meanSample);

  Serial.print("Standard Deviation: ");
  Serial.println(sdDev);  // Square-root of the mean of the squared deviations
}


void loop() {}

Results:

meanSample: 1082.05
Standard Deviation: 31.09