 # Decimal Precision

I have a variable declared as float. How do i get it to use (4) decimal places in a calculation?

If I print, for example, SerialDebug.println (variable,4);

I get 1.0000 or .9700.

Thanks,

Ken

You can't floats don't work that way.

Mark

In computations the entire precision is always used every time. When displaying using println it may not display everything unless you tell it to...thus the 4. Print uses a default precision unless you override it. The value is no less precise, just the display representation of it can be.

I do not see the results of using this precision variable after a calculation (I'm printing a graph).

If I calculate by hand (a TI calculator) I get the desired change??

I get the printing concept BTW.

Can you give a specific example?

I do not see the results of using this precision variable after a calculation

Can you please post a working example program that shows the problem ? It does not have to be your full program, but it should be possible to provide an example in very few lines of code

If I print, for example, SerialDebug.println (variable,4);

I get 1.0000 or .9700.

I don't understand the problem. Those [u]are[/u] the values of those variables, represented to four decimal places precision.

ex . . . 25.75/25.67 = 1.0031

not 1.0000 which is what I get when I read (2) values and divide!

where is the precision?

I am going through the code trying to understand the calculations better.

Will post then

Thanks,

not 1.0000 which is what I get when I read (2) values and divide!

Then you are either doing the read incorrectly, or the divide incorrectly.

This program prints the correct answer:

``````void setup() {
float a=25.75;
float b=25.67;
Serial.begin(9600);
Serial.println(a/b,4);
}
void loop(){}
``````

Another issue has creeped up for me.

I generate a calibration factor in Setup and confirm it is working & valid (locally), but it does not pass to the

loop where it is used. should I store it in EEPROM or is there a way to preserve it globally?

Thanks,

Move these to prior to setup() to make them global.

float a=25.75; float b=25.67;

Reefhermit: I have a variable declared as float. How do i get it to use (4) decimal places in a calculation? If I print, for example, SerialDebug.println (variable,4); I get 1.0000 or .9700. Thanks,

Floating point numbers are not decimal. They are a combination of a 51-bit binary number and an 11-bit power of 2 (and a sign bit and some status flags). This means that they cannot accurately represent fractions other than powers of two.

This is a well-known feature of using floating point numbers. As a result, the more calculations you perform in a series, the worse the potential errors get. There is a great deal of work in the field of numerical analysis - working out how much errors are likely to creep into commonly-needed mathematical processes when using floats, and working out ways to calculate things that will introduce the least error.

If you definitely need to calculate with four places exactly (typically, to match other hardware, or because you are working with time or currency), then multiply everything by 10000 and use integer types (short, int, long, long long).

If you simply want to display the result with four digits, you'll need a little code.

In general, always treat a floating point number as an approximation of some quantity, an approximation accurate about 15 digits. If you need more precision, then you will need some sort of extended precision math library.

There aren’t any 64 bit floating point datatypes in Arduino IDE. floats are 32 bit so you only get 7 digits of precision including the digits in front of the decimal point. float and double are the same thing in Arduino IDE.