Go Down

Topic: decimal calculation correction problem (Read 1 time) previous topic - next topic

Midway

hi,

this is the code:
Code: [Select]
#define MF 38.9
Na = Freq + MF;


When i run the software in the Nano, I see that the decimal MF is 38.9000015258 and not 38.9.

how can i decrease this "15258" to 38.9

i have tried to:
Code: [Select]
MFcorr = MF - 15258

but not work well


i appreciate all the help
ed

Erdin

Use this:
http://www.binaryconvert.com/result_float.html
Fill in "38.9" for the decimal, and click convert.
It is converted to a binary, but that binary is: 3.890000152587890625E1

It is a 32-bit floating point number, and that is the accuracy you get.

majenko


hi,

this is the code:
Code: [Select]
#define MF 38.9
Na = Freq + MF;


When i run the software in the Nano, I see that the decimal MF is 38.9000015258 and not 38.9.

how can i decrease this "15258" to 38.9

i have tried to:
Code: [Select]
MFcorr = MF - 15258

but not work well


i appreciate all the help
ed

Simple: don't use floats.  Use integer maths instead.

Instead of using the actual value, use the "milli" value.  For instance, instead of volts (3.2V) use millivolts (3200mV).  Instead of degrees C (25.6 DegC), use milliDegrees C (25600 mDegC).  Instead of Hz (143.2Hz) use milliHertz (143200mHz)...
Get 10% off all 4D Systems TFT screens this month: use discount code MAJENKO10

GoForSmoke


Simple: don't use floats.  Use integer maths instead.

Instead of using the actual value, use the "milli" value.  For instance, instead of volts (3.2V) use millivolts (3200mV).  Instead of degrees C (25.6 DegC), use milliDegrees C (25600 mDegC).  Instead of Hz (143.2Hz) use milliHertz (143200mHz)...


+1

Instead of measuring in decimal-point meters that would be measuring in millimeters.
If you need 6 places then use micrometers (millions of a meter).

Just watch your variables.
int is good -32768 to 32767 which if all 9's gives you 4 places max (9999).
long is good -4294967296 to 4294967295 gives you 9 places of 9's (999999999).
long long (64 bit integer) gives you 19 places.

Put the decimal point in when you print the number out to keep your accuracy.
1002 mm = 1.002 m

1002 / 1000 = 1 .... print 1 then print .

1002 % 1000 = 2 
2 / 100 = 0 .... print 0

2 % 100 = 2
2 / 10 = 0 .... print 0

2 / 1 = 2 .... print 2

That operation can be done in a while loop or a for loop.

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

PeterH


When i run the software in the Nano, I see that the decimal MF is 38.9000015258 and not 38.9.

how can i decrease this "15258" to 38.9


I agree with the suggestions to use fixed-point arithmetic instead of floats. However, the original problem you described is not really about the value of the expression - it is only about how that value is represented as a string. There are various ways to control how floating point numbers, and which options are available to you will depend on how you're doing that representation - which you haven't shown us.
I only provide help via the forum - please do not contact me for private consultancy.

Coding Badly


The long answer... http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

GoForSmoke

I'll stick with Kernighan and Plauger: "Floating point numbers are like piles of sand; every time you move one you lose a little sand and pick up a little dirt."

http://introcs.cs.princeton.edu/java/91float/

Quote
If the errors occur at random, we might expect a cumulative error of sqrt(N) ?m. However, if we are not vigilant in designing our numerical algorithms, these errors can propagate in very unfavorable and non-intuitive ways, leading to cumulative errors of N ?m or worse.


When I worked on any code that involved money; billing, payroll, taxes, etc, I stayed away from floating point. When I coded for NC and CNC cutting machines, I stayed away from floating point.

Besides, Arduino has no FPU. Floating point crawls without an FPU. Even 64-bit integers is faster.


I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Midway

#7
Aug 12, 2013, 09:22 pm Last Edit: Aug 12, 2013, 09:27 pm by Midway Reason: 1
Hi all,

this is the actual plan:

To tune a TV-tuner with I2C i need to calculate HEX "lsb" and "msb".

When i adjust the frequency in steps (MHz or KHz) the following code i have made to calculate the HEX adress MSB and LSB.

Xtal: DEC 16.0 is used with Xtal_a = 4.0 MHz or DEC 20 is used with Xtal_b 3.2 MHz.
MF: Mid Freqency DEC 38,9 MHz


Code: [Select]
float Na;
float Freq;
int Nb;
int Nc;
int msb;
int lsb;

Na =Freq+MF;
Nb =Na*Xtal;
Nc =((INT)(Nb/256)) * 256;
lsb =Nb-Nc;
msb =(INT)(Nb/256);


-------------------------
this is the actual reading when the software is running.

the selected frequency = 1 MHz (also with a strange number at the end "1192")

1.0000001192 = Freq
38.9000015258 = MF
16.0000000000 = Xtal
91 = Band
39.9000015258 = Na
638 = Nb
512 = Nc
7E = lsb DR2
2 = msb DR1


hope u have some more information regarding the MF correction

ed

KeithRB

There is no correction. Just like when you convert 1/3 to decimal you get 0.3333333333..., there is no exact binary conversion from 38.9 base 10 to binary. You have to live with it or use integers, as has been pointed out. Welcome to computer science!

Do you have a datasheet for the tuner?

Coding Badly

When i adjust the frequency in steps (MHz or KHz) the following code i have made to calculate the HEX adress MSB and LSB.


That code also truncates the value.  The "strange numbers" at the end can be ignored because they never become part of the final value.

GoForSmoke

Quote

the selected frequency = 1 MHz (also with a strange number at the end "1192")

1.0000001192 = Freq


The strange number is .0000001192, which in your case is too small to affect the results.
Even if Freq equaled .9999998808 it wouldn't make much difference.

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Midway

hi,

thanks for getting the Info.

When i go 20 Mhz higher this is the result:

20.0000400543 = Freq
38.9000015258 = MF
16.0000000000 = Xtal
91 = Band
58.9000396728 = Na
942 = Nb
768 = Nc
AE = lsb DR2
3 = msb DR1

The result when i go to 30 MHz:

30.0000782012 = Freq
38.9000015258 = MF
16.0000000000 = Xtal
91 = Band
68.9000778198 = Na
1102 = Nb
1024 = Nc
4E = lsb DR2
4 = msb DR1

this is only an example when it count from 1 MHz to 30 Mhz.

At this moment the actual frequency's (when selected) must be:
Code: [Select]
#define LB 0xA1                   // Low frequency band 46.25 - 170 MHz = 0xA1
#define MB 0x91                   // Mid frequency band Mid 170 - 450 MHz = 0x91
#define HB 0x31                   // High frequency band High 450 - 860 MHz = 0x31


In my opinion it make much difference.

ed


Delta_G

I don't understand the difference you think it makes.  When you make the conversion at the end, the extra part after the decimal is being truncated.  So you're getting the same answer in the end that you would have gotten with the exact numbers.

GoForSmoke

Midway, I wonder if you understand math or numbers?

20.0000400543 / 20 = 1.000002002715, the error is digits a float can't store but the print generates. A float (Arduino double and float are THE SAME) is only good to 6 or 7 digits which means you don't have 20.0000400543 but actually 20.00004. But SO WHAT?

30.0000782012 < 30.001 < 30.1 < 31.0

You have ranges like 46.25 to 170. 46.250099999 is less than 46.26 let alone 170!

The error is not significant. Where the digits are is more important than what the digits are.

If you need exact then use fixed-place integers.

And just BTW, I hope that you don't expect to measure 46.25 MHz and faster with an Arduino that runs at 16 MHz.
I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Go Up