Math / rounding error ...

Projects Programming
1

Hi all,

I've been trying to code a function to convert a large number (ie. 12345678) into BCD and have started by trying to divide the number into bytes as follows:

#include <math.h>

void setup()
{
  Serial.begin(9600);
}

byte calcFreq(unsigned long freqIn){
  byte bcdByte;
  for (int i=6; i>-1; i--) {
    if ((i==6) || (i==4) || (i==2) || (i==0)){  // only take even values, to allow pseudo BCD bytes
      Serial.print("freqIn: ");
      Serial.print(freqIn);
      bcdByte = freqIn / pow(10,i);
      freqIn = (freqIn - (bcdByte * (pow(10,i))));
      Serial.print(" bcdByte");
      Serial.print(i);
      Serial.print(" :");
      Serial.print(bcdByte);
      Serial.print(" new freqIn: ");
      Serial.println(freqIn);
    }
  }
  return bcdByte; // invalid at the moment
}

void loop()
{
  Serial.println(calcFreq(12345678));  
  delay(5000);
 }

The above (calcFreq) function is not yet complete (ie. the return portion at least).

I am outputting the data to serial for debugging purposes and I have been receiving extremely frustrating output so far:

freqIn: 12345678 bcdByte6 :12 new freqIn: 345682
freqIn: 345682 bcdByte4 :34 new freqIn: 5682
freqIn: 5682 bcdByte2 :56 new freqIn: 82
freqIn: 82 bcdByte0 :82 new freqIn: 0
82

I can not understand what is happening to the freqIn value (which started as: 12345678) and gets truncated(?) to: 345682 where I was expecting 345678

What I am trying to do is take the 12345678 and separate it out as: 0x12, 0x34, 0x56, 0x78

Can anyone give me some guidance on this issue? This code is related to a problem I am having as seen in this thread.

2

You can combine this bit:

for (int i=6; i>-1; i--) {
    if ((i==6) || (i==4) || (i==2) || (i==0)){  // only take even values, to allow pseudo BCD bytes

into a more simple:

for (int i=6; i>-1; i-=2) {

which will decrement i by 2 each time instead of one.

Where you are doing pow(10,i) it would be more efficient to do that only once and assign the result to a variable. It's faster to get the contents of a variable than to calculate a power.

On the face of it at the moment I can't see any reason for your error.

3

I would suggest converting the long to a string and bcd encoding each character...

4

pow() does not calculate what you think it does - either by design or because it's just plain Arduinofied.

pow(10,6) returns 999999
pow(10,4) returns 9999
pow(10,2) returns 99
pow(10,0) returns 1

You might do better with a simple lookup table:

const unsigned long powers[] = {1,10,100,1000,10000,100000,1000000};

Then using that in place of the pow() function.

You could reduce the number of elements and do a /2 on the counter to select it if you want to save space but use more clock cycles to do the lookup.

5

Here's a complete one for you:

const unsigned long powers[] = {1,10,100,1000,10000,100000,1000000};

void setup()
{
  Serial.begin(9600);
}

byte calcFreq(unsigned long freqIn){
  byte bcdByte;
  for (int i=6; i>-1; i-=2) {
      Serial.print("freqIn: ");
      Serial.print(freqIn);
      bcdByte = freqIn / powers[i];
      freqIn = (freqIn - (bcdByte * powers[i]));
      Serial.print(" bcdByte");
      Serial.print(i);
      Serial.print(" :");
      Serial.print(bcdByte);
      Serial.print(" new freqIn: ");
      Serial.println(freqIn);
  }
  return bcdByte; // invalid at the moment
}

void loop()
{
  Serial.println(calcFreq(12345678));  
  delay(5000);
 }

Uses the improved for(...) and an array of powers. Also, you don't need math.h any more.

All you need to do now is convert that "bcdByte" into actual BCD :wink:

6

pow() does not calculate what you think it does - either by design or because it's just plain Arduinofied.

The behaviour is neither by design nor specific to the Arduino. It is a side-effect of the way floating-point numbers work. Copious details are available here... What Every Computer Scientist Should Know About Floating-Point Arithmetic

7

Some great (and quick!) replies here.
Thanks all for the input - I am learning a lot with each comment.
I am really interested about the pow function and the reason it returns (eg.) 9999 so looks like I have some reading ahead :).

Thanks again everyone for your help - it is greatly appreciated!

8

I am really interested about the pow function and the reason it returns (eg.) 9999 so looks like I have some reading ahead

Start with looking at the argument types. The function is returning 9999.9999 which you truncate to 9999.