I recently learned that a double-precision floating point variable can't precisely hold the decimal value 0.1 and found several clear explanations of it on the web. One would expect to see something like 1.00000000000000005551e-1 when that variable is displayed to 20 digits past the decimal point. Unfortunately for me, the first platform I decided to test it on, an Arduino MKR Zero, seemed to contradict what I had just learned. Here is my code and its output:
I was expecting 555 at the end, so I'm wondering if my test is flawed. My colleague thinks that Arduino is displaying the excess significant digits incorrectly since 0 is being printed for them all. Searching through this forum I found this very informative topic: Double precision operations. Taken together, @jremington's post 19 and @westfw's post 31 suggest that this behavior might indeed be a flaw in System.out.println's implementation. Is that true? Is the output incorrect? If so, what was the rationale for coding this behavior?
What gives me pause is that I get similar results with Java and Excel, but again, my tests might be flawed. C#, Pd64, and SuperCollider all display the same non-zero digits past the precision limit.
When printing a floating point number, there is a lot of math involved to convert to decimal and separate the digits.
In every FP operation you loose some (or more) precision, so after many math operations there might only be 10 significant digits correct of the 16 you have
A simplified example, if you have 2 floating point numbers the accuracy is a half bit of the mantissa (significant) of either. If you add them something happens.
Lets assume you have 5 significant digits (to simplify the problem)
A = 1.2463
B = 1000.9
C = A + B = 1002.1463 ==> 1002.1 so you loose the last bit of A
It gets worse when A and B differ more.
A = 1.2463
B = 1.0000e6 (one million)
C = A + B = 1000001.23463 ==> still 1.0000e6 (one million) as A gets completely lost.
So if you need to add up an array of floating point values that differ a lot in magnitude, you need to sort them first and add them from abs(smallest) to abs(largest). Then the damage is as low as you can get.
If you understand the above, note that the printing in Serial.print() does one digit at a time, subtract that digit, divide by 10, and up to the next digit. Printing with 20 decimals will have 20 subtractions and divisions runtime. So in potential a lot of bits can get lost.
On a PC with a math coprocessor, the double (64 bit) math is expanded internally to IIRC 80 bit, and when the math is done the result is rounded and stored.
See - Extended precision - Wikipedia
One would expect a double, with a precision of 15 to 17 digits, to not give a precise output to 21 digits
Agreed, which is why I hedged a little with the words "something like". But look at the result I actually got, which is indistinguishable from accuracy to 21 digits! See why I'm scratching my head?
Edit: 18 hours and many responses later, my comment above doesn't make sense to me anymore. It's the underlying double that determines the precision, not the decimal representation of it. Maybe what I should have written is "I don't get why there's not random garbage in the digits beyond the precision limit" but I may be digging myself a deeper hole.
On a PC with a math coprocessor, the double (64 bit) math is expanded internally to IIRC 80 bit, and when the math is done the result is rounded and stored.
Ah, OK, this might explain why it would be unreasonable to expect the MKR Zero to output my expected 20 digits (the one that ends with 555), although I'm not sure because I don't really understand the conversion algorithm. That's less important to me than understanding the result I'm getting though.
...note that the printing in Serial.print() does one digit at a time, subtract that digit, divide by 10, and up to the next digit. Printing with 20 decimals will have 20 subtractions and divisions runtime. So in potential a lot of bits can get lost.
Makes sense, but it's not possible that the errors happened to add up to 0.10000000000000000000, is it? If I format to 40 digits, it's zeroes all the way to the right, so something else must be the cause, no?
The exact decimal presentation of the binary approximation of 0.1 is the result of the print algorithm. It applies only to numbers less than one, where all the bits of the significand are dedicated to the fractional part, so that when "the math" is applied, the results round "as expected".
With 1.1, you "lose" a bit to represent the integer part, so the remaining bits are no longer as close to 0.1 as possible; the difference no longer "rounds out", and you see the "actual" binary value.
That's the hand-wavy explanation Do you need more?
BTW, obligatory plug for float.exposed, useful for this topic
Did you read the link(s) in my post that you referenced?
"Correct" printing of floating point values is "hard", and expensive.
Although, I'm not sure why you expect 20 digits of correct output from a format with fewer digits of actual precision.
It is possible and it depends on the print algorithm as @kenb4 pointed out.
If you want to understand it you can add HEX dump statements into the float / double printing algorithm to study what happens with the intermediate values / variables. This investigation will cost you quite some time but you will learn how it works in detail.
That's great, thank you for the link to the print algorithm. I didn't know it was accessible. Hand waving is plenty for now--I need to dig in to the code and try to understand what's going on. I think float.exposed will be really useful for that.
Did you read the link(s) in my post that you referenced?
I'm sorry, I didn't, I've put it on my reading list.
I'm not sure why you expect 20 digits of correct output from a format with fewer digits of actual precision.
I think this in part explains my misunderstanding. I only learned yesterday which routine was responsible for that output and that it was accessible to me. On a quick glance it looks like it would require more than double precision to compute the digits beyond the precision limit, is that correct? I'm provisionally concluding that partly because on an Arduino with single-precision doubles, displaying (float)0.1 to 10 digits also doesn't display the actual binary value. Please let me know if I'm on the right track.
Also, the fact that you call it "correct output" really gets to what started me on this topic originally--I am reacting to my colleague calling Arduino's output wrong. But I think I know what he means now.
I haven't bothered to look at the Arduino .print() function in detail, but the algorithm used clearly "makes up" decimal digits, if asked to print more than the precision allowed by the float representation. Those digits cannot be taken seriously for the following reason:
that a double-precision floating point variable can't precisely hold the decimal value 0.1
The binary representation is an infinite, repeating series of base two fractions, so an exact base 2 representation of the decimal fraction 1/10 is not possible, similar to trying to express 1/3 or 1/7 as a decimal fraction.
Do you need more?
