 # RE: Decimal Digits of Precision

Hi Gang

The Arduino Reference states the following.

Floats have only 6-7 decimal digits of precision. That means the total number of digits, not the number to the right of the decimal point.

This is a contradiction. The first sentence states that a float has 6-7 decimal digits of precision. From what I understand decimal digits are to the right of the decimal point. Does this mean the following is correct?

``````const float pi = 3.1415927
``````

The second sentence states that the total number of digits, not the number to the right of the decimal point. Does this mean the following is correct?

``````const float pi = 3.141592
``````

Any help would be greatly appreciated.

Cheers

Jase :)

Why don't you just try it?

The script to test would be much shorter than your question.

Decimal means base ten. Digits are individual numeric symbols. For example 42 has two decimal digits. You misunderstand the correct definition.

ilovetoflyfpv:
Hi Gang

The Arduino Reference states the following.

This is a contradiction. The first sentence states that a float has 6-7 decimal digits of precision. From what I understand decimal digits are to the right of the decimal point.

Wrong. Digits precision is counting across all(!) digits, starting with the first non-zero digit in the number:

First non-zero digit in number 0003.14159 is the digit “3”, so there is one digit before and 5 digits after the decimal point = total of “six significant digits”, and that’s pretty much all which the ‘float’ type can provide in decimal number precision.

IEEE single precision floating point is represented in a 32-bit wide number like so: ( 1 sign bit, 8 exponent bits, 23 fraction bits ), calculated in this manner:

float = (-1)^sign * (1 + fraction) * 2^(exponent – bias)

where single-precision bias is 127, which ensures the exponent is unsigned.

Since the largest number 23 bits can represent is, 8388608, you can see the maximum precision will be in the order of 6 or 7 decimal digits.