Go Down

Topic: RE: Decimal Digits of Precision (Read 1 time) previous topic - next topic

ilovetoflyfpv

Hi Gang

The Arduino Reference states the following.

Quote
Floats have only 6-7 decimal digits of precision. That means the total number of digits, not the number to the right of the decimal point.
This is a contradiction. The first sentence states that a float has 6-7 decimal digits of precision. From what I understand decimal digits are to the right of the decimal point. Does this mean the following is correct?

Code: [Select]
const float pi = 3.1415927

The second sentence states that the total number of digits, not the number to the right of the decimal point. Does this mean the following is correct?

Code: [Select]
const float pi = 3.141592

Any help would be greatly appreciated.

Cheers

Jase :)

Whandall

Why don't you just try it?

The script to test would be much shorter than your question.
Ah, this is obviously some strange usage of the word 'safe' that I wasn't previously aware of. (D.Adams)

aarg

Decimal means base ten. Digits are individual numeric symbols. For example 42 has two decimal digits. You misunderstand the correct definition.
  ... with a transistor and a large sum of money to spend ...
Please don't PM me with technical questions. Post them in the forum.

jurs

Hi Gang

The Arduino Reference states the following.

This is a contradiction. The first sentence states that a float has 6-7 decimal digits of precision. From what I understand decimal digits are to the right of the decimal point.
Wrong. Digits precision is counting across all(!) digits, starting with the first non-zero digit in the number:

First non-zero digit in number 0003.14159 is the digit "3", so there is one digit before and 5 digits after the decimal point = total of "six significant digits", and that's pretty much all which the 'float' type can provide in decimal number precision.

#4
Mar 29, 2017, 02:49 pm Last Edit: Mar 29, 2017, 02:50 pm by JimEli
IEEE single precision floating point is represented in a 32-bit wide number like so: ( 1 sign bit, 8 exponent bits, 23 fraction bits ), calculated in this manner:

  float =  (-1)^sign * (1 + fraction) * 2^(exponent - bias)

where single-precision bias is 127, which ensures the exponent is unsigned.

Since the largest number 23 bits can represent is, 8388608, you can see the maximum precision will be in the order of 6 or 7 decimal digits.

To answer your question specifically, 3.1415927.


You can play with conversions here.




ilovetoflyfpv

Hi Gang

Thanks for the replies. I think I get it.

Cheers

Jase :)

Go Up