Go Down

Topic: Why won't this convert properly?! It is driving me maddddd (Read 901 times) previous topic - next topic

PeterMHull

Apr 12, 2013, 07:29 am Last Edit: Apr 12, 2013, 07:50 am by PeterMHull Reason: 1
So I've tried converting the data in the following code both using casting and conversion methods.
i.e.  int(variable) and (int) Variable

The offending code has been isolated to this portion, which i stripped down from the rest of the project:

void loop()
{
 double total = 0;
 int y = 0;

 for(int x = 0; x <= 7; x++)
 {
   while(y<=x)
   {
   total += pow(2,x);
   y++;
   }
   Serial.print("Unconverted total (double Type): ");
   Serial.println(total);
   int intTotalCast = (int)total;
   int intTotalConversion = int(total);

   Serial.print("Converted using casting (int Type): ");
   Serial.println(intTotalCast);
   Serial.print("Converted using Converter (int Type): ");
   Serial.println(intTotalConversion);
   delay(1000);
   Serial.println("Loop Starts Over");
 }  
Here's the output:

Unconverted total (double Type): 1.00
Converted using casting (int Type): 1
Converted using Converter (int Type): 1
Loop Starts Over
Unconverted total (double Type): 3.00
Converted using casting (int Type): 3
Converted using Converter (int Type): 3
Loop Starts Over
Unconverted total (double Type): 7.00
Converted using casting (int Type): 6
Converted using Converter (int Type): 6
Loop Starts Over
Unconverted total (double Type): 15.00
Converted using casting (int Type): 14
Converted using Converter (int Type): 14
Loop Starts Over
Unconverted total (double Type): 31.00
Converted using casting (int Type): 30
Converted using Converter (int Type): 30
Loop Starts Over
Unconverted total (double Type): 63.00
Converted using casting (int Type): 62
Converted using Converter (int Type): 62
Loop Starts Over
Unconverted total (double Type): 127.00
Converted using casting (int Type): 126
Converted using Converter (int Type): 126
Loop Starts Over
Unconverted total (double Type): 255.00
Converted using casting (int Type): 254
Converted using Converter (int Type): 254


As you can see, the converted values start off correct, 1:1, 3:3, then it goes weird... 7:6, 15:14, 31:30, etc...

every single value after 3 is off by one, i don't get it!!!! Please help, I'm out of ideas.

BTW, what I'm trying to do basically... is count up in binary... to get this pattern:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111

Edit:
Cleaned it up, made it a little easier to read. Hopefully.

WizenedEE

Because of the way doubles are stored, they can't be perfectly precise. So when you type 5 it might hear 5.00000012 or 4.9999984. So while 5.000000012 casted to an int (which is always rounded down) may be 5, 4.99999984 casted to an int is 4, which is obviously incorrect.

So instead of rounding down, you want to round to nearest. The easiest way to do this is to add 0.5 and then round down:
Code: [Select]

int intTotalCast = (int)(total+.5);


See if that helps you

PeterMHull

Thanks, that worked.  Even with your explanation I don't understand how that could happen.  I mean, i would think a data type of double would imply even  more precision, rather then less.  I would expect it to store 5.00000000000000000000. Also, I assumed it would simply truncate the value, so that everything to the right of the decimal point would be chopped off and discarded.   It seems silly... but it worked! thanks again.

WizenedEE

Thanks, that worked.  Even with your explanation I don't understand how that could happen.  I mean, i would think a data type of double would imply even  more precision, rather then less.

Nope. Fixed point types (int, long, byte) always have more precision than floating point (float, double) of the same length since they operate over a much smaller range of numbers. But even with ten thousand digits after the decimal point of precision, you can't store precisely 5 the way arduino does, just like you can't specify 1/3 precisely in decimal (it's ABOUT 0.3333333). And even if it's 0.00000000000000000001 below 5, it just chops off the stuff after the decimal point leaving you with four.
Quote
Also, I assumed it would simply truncate the value, so that everything to the right of the decimal point would be chopped off and discarded.

it does. That's why 4.99999999999 makes 4.

AWOL

If you want powers of two, why are you using floating point types at all?
The processor is perfectly happy working in binary.
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

PeterMHull

Quote
it does. That's why 4.99999999999 makes 4.


ok, but why does it return 5.00 when displaying it's double form?  It rounds for println purposes?

Quote
If you want powers of two, why are you using floating point types at all?
The processor is perfectly happy working in binary.


I'm playing with someone elses code and that's the datatype they have the class setup to accept.

WizenedEE

#6
Apr 12, 2013, 08:48 am Last Edit: Apr 12, 2013, 08:50 am by WizenedEE Reason: 1

Quote
it does. That's why 4.99999999999 makes 4.

ok, but why does it return 5.00 when displaying it's double form?  It rounds for println purposes?

Yes, the implementation of print(double) uses the same trick I showed you. From Print.cpp:
Code: [Select]

size_t Print::printFloat(double number, uint8_t digits)
{
 size_t n = 0;
 
 if (isnan(number)) return print("nan");
 if (isinf(number)) return print("inf");
 if (number > 4294967040.0) return print ("ovf");  // constant determined empirically
 if (number <-4294967040.0) return print ("ovf");  // constant determined empirically
 
 // Handle negative numbers
 if (number < 0.0)
 {
    n += print('-');
    number = -number;
 }

 // Round correctly so that print(1.999, 2) prints as "2.00"
 double rounding = 0.5;
 for (uint8_t i=0; i<digits; ++i)
   rounding /= 10.0;
 
 number += rounding;

 // Extract the integer part of the number and print it
 unsigned long int_part = (unsigned long)number;
 double remainder = number - (double)int_part;
 n += print(int_part);

 // Print the decimal point, but only if there are digits beyond
 if (digits > 0) {
   n += print(".");
 }

 // Extract digits from the remainder one at a time
 while (digits-- > 0)
 {
   remainder *= 10.0;
   int toPrint = int(remainder);
   n += print(toPrint);
   remainder -= toPrint;
 }
 
 return n;
}


if you're still curious:
http://en.wikipedia.org/wiki/Double-precision_floating-point_format

PeterMHull

Alright, I see what your saying.  I'll give that link a look tomorrow, thanks.

Coding Badly


And if you're really curious...
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Fletcher Chr

Hi

Quote
BTW, what I'm trying to do basically... is count up in binary... to get this pattern:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111


How about this:

Code: [Select]
void setup(){
  Serial.begin(115200);
  int lines=8;
  int number=1;
 
  for (int s=lines-1;s>=0;s--){
    for (int t=lines-1;t>=0;t--){
      Serial.print(bitRead(number,t));
    }
    Serial.println();
    number<<=1;  // bitshift number one to the left
    number+=1;   // add 1 to the right most bit
  }
}

void loop(){}


Output:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111


-Fletcher

AWOL

Quote
what I'm trying to do basically... is count up in binary... to get this pattern:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111
which of course is unary, not binary.
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

Delta_G


Quote
what I'm trying to do basically... is count up in binary... to get this pattern:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111
which of course is unary, not binary.


Can you explain what you mean by that?  They look like binary numbers to me...

AWOL

Well, of course they're binary numbers.
Just not consecutive binary numbers.
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

Go Up