Go Down

Topic: Floating point math issues....last significant digit not what I expected (Read 599 times) previous topic - next topic

jerseyguy1996

Hello!  Given the following code:

Code: [Select]
char latit[10] = {"4041.2358"};  //Degrees = 40, Minutes = 41, Seconds = 23.58
Serial.print("latit = ");
Serial.println(latit);

float flatit = atof(latit);
Serial.print("flatit = ");
Serial.println(flatit, 6);

flatit /= 100.00;
Serial.print("flatit = ");
Serial.println(flatit, 6);

float flatitdegrees = (int)flatit; //isolate degrees
float temp = (flatit - (int)flatit) * 100;
float fminutes = (int)(temp); //isolate minutes
float fseconds = (temp - fminutes) * 100; //isolate seconds

Serial.print("Degrees = ");
Serial.println(flatitdegrees);
Serial.print("Minutes = ");
Serial.println(fminutes,2);
Serial.print("Seconds = ");
Serial.println(fseconds, 2);


I get an output that looks like this:

Code: [Select]
latit = 4041.2358
flatit = 4041.235839
flatit = 40.412357
Degrees = 40.00
Minutes = 41.00
Seconds = 23.57


Can someone help me to understand what is going on with those last significant digits?
Arduino Uno;
Mega328

PaulS

A float has between 6 and 7 digits of precision. You string has 8. Converting it to a float does not produce and exact value, with 8 digits of precision.

Using Serial.print() to print more digits does not result in more precision, just more guesswork.

The results you are getting are exactly what I'd expect.

robtillaart

floating points are just not as precise as you want them to be.

The Arduino (UNO) uses IEEE-754 float format = 32 bits
1 bit for sign
23 bit for the fraction
8 bit for exponent (powers of 2)

the fraction part can only hold about 7 digits precision. YOur number has 8 so it is rounded to the nearest number it can represent.

Details see Wikipedia - http://en.wikipedia.org/wiki/IEEE_754-1985 -
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

jerseyguy1996

Thanks everyone for the great explanations.  Quick and easy fix in my code was to separate the digits before and after the decimal using the decimal as the delimiter and then working the math on each section individually.  Works perfect now!
Arduino Uno;
Mega328

dhenry

Quote
Floating point math issues....last significant digit not what I expected


It would have been a miracle if t he last significant digit were what you expected. That's the nature of floating point math.

robtillaart

Quote
That's the nature of floating point math.

And we haven't even discussed error propagation due to math operators
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Go Up