Pages: [1]   Go Down
Author Topic: decimal calculation correction problem  (Read 907 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 46
I just want to learn more :)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

hi,

this is the code:
Code:
#define MF 38.9
Na = Freq + MF;

When i run the software in the Nano, I see that the decimal MF is 38.9000015258 and not 38.9.

how can i decrease this "15258" to 38.9

i have tried to:
Code:
MFcorr = MF - 15258

but not work well


i appreciate all the help
ed
Logged

Offline Offline
Edison Member
*
Karma: 58
Posts: 2078
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Use this:
http://www.binaryconvert.com/result_float.html
Fill in "38.9" for the decimal, and click convert.
It is converted to a binary, but that binary is: 3.890000152587890625E1

It is a 32-bit floating point number, and that is the accuracy you get.
Logged

UK
Offline Offline
Faraday Member
**
Karma: 100
Posts: 4153
Where is your SSCCE?!?!
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

hi,

this is the code:
Code:
#define MF 38.9
Na = Freq + MF;

When i run the software in the Nano, I see that the decimal MF is 38.9000015258 and not 38.9.

how can i decrease this "15258" to 38.9

i have tried to:
Code:
MFcorr = MF - 15258

but not work well


i appreciate all the help
ed
Simple: don't use floats.  Use integer maths instead.

Instead of using the actual value, use the "milli" value.  For instance, instead of volts (3.2V) use millivolts (3200mV).  Instead of degrees C (25.6 DegC), use milliDegrees C (25600 mDegC).  Instead of Hz (143.2Hz) use milliHertz (143200mHz)...
Logged

Get 10% off all 4D Systems TFT screens this month: use discount code MAJENKO10

Pittsburgh, PA, USA
Offline Offline
Faraday Member
**
Karma: 98
Posts: 4813
I learn a bit every time I visit the forum.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Simple: don't use floats.  Use integer maths instead.

Instead of using the actual value, use the "milli" value.  For instance, instead of volts (3.2V) use millivolts (3200mV).  Instead of degrees C (25.6 DegC), use milliDegrees C (25600 mDegC).  Instead of Hz (143.2Hz) use milliHertz (143200mHz)...

+1

Instead of measuring in decimal-point meters that would be measuring in millimeters.
If you need 6 places then use micrometers (millions of a meter).

Just watch your variables.
int is good -32768 to 32767 which if all 9's gives you 4 places max (9999).
long is good -4294967296 to 4294967295 gives you 9 places of 9's (999999999).
long long (64 bit integer) gives you 19 places.

Put the decimal point in when you print the number out to keep your accuracy.
1002 mm = 1.002 m

1002 / 1000 = 1 .... print 1 then print .

1002 % 1000 = 2 
2 / 100 = 0 .... print 0

2 % 100 = 2
2 / 10 = 0 .... print 0

2 / 1 = 2 .... print 2

That operation can be done in a while loop or a for loop.

Logged

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

UK
Offline Offline
Shannon Member
****
Karma: 223
Posts: 12630
-
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

When i run the software in the Nano, I see that the decimal MF is 38.9000015258 and not 38.9.

how can i decrease this "15258" to 38.9

I agree with the suggestions to use fixed-point arithmetic instead of floats. However, the original problem you described is not really about the value of the expression - it is only about how that value is represented as a string. There are various ways to control how floating point numbers, and which options are available to you will depend on how you're doing that representation - which you haven't shown us.
Logged

I only provide help via the forum - please do not contact me for private consultancy.

Global Moderator
Dallas
Online Online
Shannon Member
*****
Karma: 210
Posts: 13039
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset


The long answer... http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
Logged

Pittsburgh, PA, USA
Offline Offline
Faraday Member
**
Karma: 98
Posts: 4813
I learn a bit every time I visit the forum.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I'll stick with Kernighan and Plauger: "Floating point numbers are like piles of sand; every time you move one you lose a little sand and pick up a little dirt."

http://introcs.cs.princeton.edu/java/91float/

Quote
If the errors occur at random, we might expect a cumulative error of sqrt(N) εm. However, if we are not vigilant in designing our numerical algorithms, these errors can propagate in very unfavorable and non-intuitive ways, leading to cumulative errors of N εm or worse.

When I worked on any code that involved money; billing, payroll, taxes, etc, I stayed away from floating point. When I coded for NC and CNC cutting machines, I stayed away from floating point.

Besides, Arduino has no FPU. Floating point crawls without an FPU. Even 64-bit integers is faster.


Logged

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Offline Offline
Newbie
*
Karma: 0
Posts: 46
I just want to learn more :)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi all,

this is the actual plan:

To tune a TV-tuner with I2C i need to calculate HEX "lsb" and "msb".

When i adjust the frequency in steps (MHz or KHz) the following code i have made to calculate the HEX adress MSB and LSB.

Xtal: DEC 16.0 is used with Xtal_a = 4.0 MHz or DEC 20 is used with Xtal_b 3.2 MHz.
MF: Mid Freqency DEC 38,9 MHz


Code:
float Na;
float Freq;
 int Nb;
 int Nc;
 int msb;
 int lsb;

Na =Freq+MF;
Nb =Na*Xtal;
Nc =((INT)(Nb/256)) * 256;
lsb =Nb-Nc;
msb =(INT)(Nb/256);

-------------------------
this is the actual reading when the software is running.

the selected frequency = 1 MHz (also with a strange number at the end "1192")

1.0000001192 = Freq
38.9000015258 = MF
16.0000000000 = Xtal
91 = Band
39.9000015258 = Na
638 = Nb
512 = Nc
7E = lsb DR2
2 = msb DR1


hope u have some more information regarding the MF correction

ed
« Last Edit: August 12, 2013, 02:27:56 pm by Midway » Logged

Offline Offline
Edison Member
*
Karma: 33
Posts: 1468
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

There is no correction. Just like when you convert 1/3 to decimal you get 0.3333333333..., there is no exact binary conversion from 38.9 base 10 to binary. You have to live with it or use integers, as has been pointed out. Welcome to computer science!

Do you have a datasheet for the tuner?
Logged

Global Moderator
Dallas
Online Online
Shannon Member
*****
Karma: 210
Posts: 13039
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

When i adjust the frequency in steps (MHz or KHz) the following code i have made to calculate the HEX adress MSB and LSB.

That code also truncates the value.  The "strange numbers" at the end can be ignored because they never become part of the final value.
Logged

Pittsburgh, PA, USA
Offline Offline
Faraday Member
**
Karma: 98
Posts: 4813
I learn a bit every time I visit the forum.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
the selected frequency = 1 MHz (also with a strange number at the end "1192")

1.0000001192 = Freq

The strange number is .0000001192, which in your case is too small to affect the results.
Even if Freq equaled .9999998808 it wouldn't make much difference.

Logged

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Offline Offline
Newbie
*
Karma: 0
Posts: 46
I just want to learn more :)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

hi,

thanks for getting the Info.

When i go 20 Mhz higher this is the result:

20.0000400543 = Freq
38.9000015258 = MF
16.0000000000 = Xtal
91 = Band
58.9000396728 = Na
942 = Nb
768 = Nc
AE = lsb DR2
3 = msb DR1

The result when i go to 30 MHz:

30.0000782012 = Freq
38.9000015258 = MF
16.0000000000 = Xtal
91 = Band
68.9000778198 = Na
1102 = Nb
1024 = Nc
4E = lsb DR2
4 = msb DR1

this is only an example when it count from 1 MHz to 30 Mhz.

At this moment the actual frequency's (when selected) must be:
Code:
#define LB 0xA1                   // Low frequency band 46.25 - 170 MHz = 0xA1
#define MB 0x91                   // Mid frequency band Mid 170 - 450 MHz = 0x91
#define HB 0x31                   // High frequency band High 450 - 860 MHz = 0x31

In my opinion it make much difference.

ed

Logged

Offline Offline
God Member
*****
Karma: 19
Posts: 785
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I don't understand the difference you think it makes.  When you make the conversion at the end, the extra part after the decimal is being truncated.  So you're getting the same answer in the end that you would have gotten with the exact numbers.
Logged

Pittsburgh, PA, USA
Offline Offline
Faraday Member
**
Karma: 98
Posts: 4813
I learn a bit every time I visit the forum.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Midway, I wonder if you understand math or numbers?

20.0000400543 / 20 = 1.000002002715, the error is digits a float can't store but the print generates. A float (Arduino double and float are THE SAME) is only good to 6 or 7 digits which means you don't have 20.0000400543 but actually 20.00004. But SO WHAT?

30.0000782012 < 30.001 < 30.1 < 31.0

You have ranges like 46.25 to 170. 46.250099999 is less than 46.26 let alone 170!

The error is not significant. Where the digits are is more important than what the digits are.

If you need exact then use fixed-place integers.

And just BTW, I hope that you don't expect to measure 46.25 MHz and faster with an Arduino that runs at 16 MHz.
Logged

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Pages: [1]   Go Up
Jump to: