Convert bytes to double using Arduino Mega

Hello,

I am fairly new to Arduino. I have recently started working on a basic RC boat design by following tutorials. I am using a VectorNav VN-200 GNSS system for this project. The issue I am currently facing is that the latitude, longitude and altitude are of the double type and I am having issues printing these using my Arduino Mega (I am aware of the issues with double using the Mega). So far, I have written a function that takes the bytes and converts them to double. I then extract the latitude, longitude and altitude from the message but have had no luck getting these to print. I either see 0 or ovf. The following is what I have:

// Convert byte array to double
double bytesToDouble(const unsigned char* bytes) {
    double value;
    memcpy(&value, bytes, sizeof(double));
    return value;

//

Extract poslat, poslon, and posalt (double) from bytes 45-68
        double poslat = bytesToDouble(&message[44]);  // bytes 45-52
        double poslon = bytesToDouble(&message[52]);  // bytes 53-60
        double posalt = bytesToDouble(&message[60]);  // bytes 61-68
        
        Serial.print(poslat, 6);

Can someone please help me with this or point me in the right direction? Thank you!

Double is the same as float on a Mega, so it is four bytes, not the eight that you need.

Thanks for your response!

You will have to forgive me, I don't come from a computer science/software background so a lot of this stuff is new for me. I did read that the Mega treats a double as a float. So if it's four bytes and latitude here takes up 8, how do I extract it?

Looks like there is a library, fp64lib, that implements double on the Mega. I'm not familiar with it, but might work for what you want.

Surely that allows data transfer in the form of formatted ASCII text, rather than binary.

Please explain what communications protocol you are using with that device.

If you are required to transmit 64 bit double values in binary, for the Mega you would have to convert them into the 32 bit binary equivalent, which is not trivial.

Wow, "Handcrafted 64-bit floating point routines", all in .S files: assembly. Looking through the file names, I don't see a converter, but there may be a way to do it. If not....

But a fun exercise. This should help see if you're on the right track

struct Float64LE {
  unsigned mantissH29:8;
  unsigned mantissG29:8;
  unsigned mantissF29:8;
  unsigned mantissE29:5;
  unsigned mantissD23:3;
  unsigned mantissC23:8;
  unsigned mantissB23:8;
  unsigned mantissA23:4;
  unsigned exponentB11:4;
  unsigned exponentA11:7;
  int sign:1;

  int16_t exponent() const {
    int16_t ret = exponentA11 << 4;
    ret |= exponentB11;
    return ret - 0x3FF;
  }
  uint32_t mantissa23() const {
    uint32_t ret = mantissA23;
    ret <<= 8;
    ret |= mantissB23;
    ret <<= 8;
    ret |= mantissC23;
    ret <<= 3;
    ret |= mantissD23;
    return ret;
  }
}__attribute__((packed));

struct Float32LE {
  unsigned mantissC23:8;
  unsigned mantissB23:8;
  unsigned mantissA23:7;
  unsigned exponentB8:1;
  unsigned exponentA8:7;
  int sign:1;

  void setExponent(int16_t x) {
    x += 0x7F;
    exponentB8 = x;
    x >>= 1;
    exponentA8 = x;
  }
  void setMantissa(uint32_t m) {
    mantissC23 = m;
    m >>= 8;
    mantissB23 = m;
    m >>= 8;
    mantissA23 = m;
  }
}__attribute__((packed));

float floating8to4(const byte *raw) {
#ifdef LET_THE_CHIP_DOIT
  if (sizeof(double) == 8) {
    auto d = reinterpret_cast<const double*>(raw);
    return static_cast<float>(*d);
  }
#endif
  auto f64 = reinterpret_cast<const Float64LE*>(raw);
  Float32LE f32;
  f32.sign = f64->sign;
  f32.setExponent(f64->exponent());
  f32.setMantissa(f64->mantissa23());
  Serial.println(*reinterpret_cast<uint32_t*>(&f32), HEX);
  return *reinterpret_cast<float*>(&f32);
}

void setup() {
  Serial.begin(115200);
  Serial.println(sizeof(Float64LE));
  Serial.println(sizeof(Float32LE));
  // https://float.exposed/0x404e4e8000000000
  byte buf[8] = {0 ,0, 0, 0, 0x80, 0x4e, 0x4e, 0x40};
  Serial.println(floating8to4(buf), 8);
}

void loop() {}

It's missing over/underflow if the exponent Is out of range, stuff like that.

I did not look at the library in detail, but if it uses the standard 8-byte format for a double then using memcpy with a float64_t variable should work, although it will also depend on whether the data being received is in the correct byte order.

can you post the received byte values (the entire message)?

Yes it allows ASCII data transfer, @anon94801460. I configured it to output binary data for 2 reasons - one, to learn and gain experience with parsing serial binary data and two, the ASCII packets are preset and some of them don't have time included whereas with the binary packets, I can choose what I want to output so timestamps are not an issue.

I am using serial TTL for comms between my GNSS unit and the Mega.

Understood. Given that this might take some time for me to fully understand and implement, I can switch to ASCII to make life easy. I can come back to the issue at hand at a later time. I appreciate you taking the time to respond!

@david_2018, I looked through the library and I will be honest, it will take some time for me to fully understand the technical details (I don't want to copy and paste code simply to make things work without understanding what I am doing). So for now, I think I'll switch to ASCII while continuing to educate myself on this a little more. Thank you for the advice and assistance!

Thanks for your response, @gcjr.

Sure, a received packet looks like the following:

0xFA 0x2B 0xE9 0x01 0x08 0x02 0x1A 0x80 0x02 0x00 0x09 0x00 0x68 0x47 0xB9 0xFB 0x35 0x06 0x00 0x00 0x23 0xAD 0xFF 0xC2 0xEB 0x8B 0xD7 0xC0 0x83 0xFC 0x58 0x3B 0x7C 0x6A 0x09 0x3A 0x43 0xD7 0x01 0x3B 0x00 0x30 0x5C 0xB6 0xAA 0x6E 0x6F 0x30 0x05 0x07 0x49 0x40 0x73 0xE2 0x14 0xB9 0x82 0xAE 0x5B 0xC0 0x00 0x58 0x8B 0x25 0xFB 0x1D 0x86 0x40 0x50 0x3A 0x6B 0x3C 0x82 0x0C 0x2F 0xBD 0x4C 0x9E 0x81 0xBE 0xDE 0x5B 0x91 0xBF 0xAE 0x21 0x98 0xBC 0xF7 0x9D 0x1B 0xC1 0x21 0x09 0x07 0xBB 0x5F 0xA8 0x79 0x46 0x84 0x01 0x00 0x08 0x03 0x1F 0x19 0x85 0x00 0xA4 0x48 0xA1 0xBB 0x5F 0xC3 0x19 0x3D 0x05 0x39 0x82 0xBE 0x6A 0x88

The first 12 bytes form the header. Total packet size is 123 bytes and the payload occupies 109 bytes. Latitude, longitude and altitude are 8 bytes each from bytes 45 to 68 (counted from 1 instead of from 0 since I come from a MATLAB background).

Thanks a lot, @kenb4!

This really intrigues me. Although I don't fully grasp all of it, I will study this and try to see if I can make things work. For now, to make things relatively simple for myself, I'll switch to ASCII and work with that but I'll continue to come back to this and keep at it. Appreciate you taking the time!

Be aware, limited serial buffer lengths on the Uno/Nano implementations may leave you frustrated; with ASCII vs binary, the strings will be much longer, and unless you empty the serial buffer ASAP, your GPS could overrun the buffer content, resulting in mangled transmissions.
Look for shorter command responses. I don't know the GPS stuff, probably should look sometime, but any command response over around 65 bytes will require that you start emptying the buffer before the whole response has arrived.

are these the correct lat/long

 50.054846
 -110.726729

counting from 0, location 44 - 51 and 52-59.

not sure where you get 68 from

@gcjr - those are absolutely correct! Can you please tell me how you managed to get those?

Altitude is the next 8 bytes after longitude which is what I was counting till 68 (that would be bytes 60-67 counting from 0).

Thanks for the advice, @camsysca. I was unaware of that. I will keep this in mind - learning a ton here today!

The third float64_t is the altitude, in meters apparently. Given the magnitude of lat/long, float32_t is accurate to about 2 in the sixth decimal place, which is... a little under a foot.

(ETA: using 64-bit floats wastes bogus precision. There's no way it can tell you're at 123.456789000001 and not 123.456789000002, neither degrees nor meters. If they stuck with 32-bit float, that's wider compatibility with "IoT" and trillions of bytes saved over the next 100 years. Maybe there's some higher-level languages that will deal with 64-bit floats only for interop.)

So beware that this location is now out in the internet. (Probably should have cautioned that earlier.) In Google Maps, you can just type the two numbers with a comma in between to place those coordinates.

Be sure to check out that URL at float.exposed in a comment near the bottom of that code earlier.

look this over

output

       msg  0x40 0x49 0x07 0x05 0x30 0x6f 0x6e 0xaa
  50.0548
       msg  0xc0 0x5b 0xae 0x82 0xb9 0x14 0xe2 0x73
 -110.7267
unsigned char msg [] = {
    0xFA, 0x2B, 0xE9, 0x01,  0x08, 0x02, 0x1A, 0x80,  0x02, 0x00, 0x09, 0x00,
    0x68, 0x47, 0xB9, 0xFB,  0x35, 0x06, 0x00, 0x00,  0x23, 0xAD, 0xFF, 0xC2,
    0xEB, 0x8B, 0xD7, 0xC0,  0x83, 0xFC, 0x58, 0x3B,  0x7C, 0x6A, 0x09, 0x3A,
    0x43, 0xD7, 0x01, 0x3B,  0x00, 0x30, 0x5C, 0xB6,  0xAA, 0x6E, 0x6F, 0x30,
    0x05, 0x07, 0x49, 0x40,  0x73, 0xE2, 0x14, 0xB9,  0x82, 0xAE, 0x5B, 0xC0,
    0x00, 0x58, 0x8B, 0x25,  0xFB, 0x1D, 0x86, 0x40,  0x50, 0x3A, 0x6B, 0x3C,
    0x82, 0x0C, 0x2F, 0xBD,  0x4C, 0x9E, 0x81, 0xBE,  0xDE, 0x5B, 0x91, 0xBF,
    0xAE, 0x21, 0x98, 0xBC,  0xF7, 0x9D, 0x1B, 0xC1,  0x21, 0x09, 0x07, 0xBB,
    0x5F, 0xA8, 0x79, 0x46,  0x84, 0x01, 0x00, 0x08,  0x03, 0x1F, 0x19, 0x85,
    0x00, 0xA4, 0x48, 0xA1,  0xBB, 0x5F, 0xC3, 0x19,  0x3D, 0x05, 0x39, 0x82,
    0xBE, 0x6A, 0x88
};

char s [90];

// -----------------------------------------------------------------------------
void dispHex (
    byte       *p,
    int         nByte,
    const char *txt )
{
    sprintf (s, "  %8s ", txt);
    Serial.print (s);

    for (int i = nByte -1; i >= 0; i--)  {
        sprintf (s, " 0x%02x", p [i]);
        Serial.print (s);
    }
    Serial.println ();
}

// -----------------------------------------------------------------------------
float
dblToFlt (
    long long lld )
{
    long long llE;
    long long llF;
    long long ll   = 0;

    memcpy (& llE, & lld, 8);
    memcpy (& llF, & lld, 8);

    byte sgn    =  lld >> 63  & 1;
    byte expSgn = (lld >> 62) & 1;;
    
    llE   = lld >> 52;
    llE   = llE & ((1 << 7) -1);
    llE <<= 23;
    llE  |=    sgn * (1LL << 31);
    llE  |= expSgn * (1LL << 30);

    llF  = lld >> 29;
    llF &= (1L << 23) -1;
    ll  = llE | llF;

    float     f;
    memcpy (& f, &ll, 4);

    return f;
}

// -----------------------------------------------------------------------------
float getDbl (
    int idx )
{
    dispHex (& msg [idx], 8, "msg");

    long long lld;
    memcpy (&lld, & msg [idx], 8);

    float f = dblToFlt (lld);

    return f;
}

// -----------------------------------------------------------------------------
void setup ()
{
    Serial.begin (9600);

    char   t [10];
    float longitude = getDbl (44);

    dtostrf (longitude, 8, 4, t);
    sprintf (s, " %s", t);
    Serial.println (s);

    float latitude = getDbl (52);

    dtostrf (latitude, 8, 4, t);
    sprintf (s, " %s", t);
    Serial.println (s);
}

void loop () {}

A slightly different bit of code to accomplish the same thing as @gcjr .

const byte data[] = {
  0xFA, 0x2B, 0xE9, 0x01, 0x08, 0x02, 0x1A, 0x80,
  0x02, 0x00, 0x09, 0x00, 0x68, 0x47, 0xB9, 0xFB,
  0x35, 0x06, 0x00, 0x00, 0x23, 0xAD, 0xFF, 0xC2,
  0xEB, 0x8B, 0xD7, 0xC0, 0x83, 0xFC, 0x58, 0x3B,
  0x7C, 0x6A, 0x09, 0x3A, 0x43, 0xD7, 0x01, 0x3B,
  0x00, 0x30, 0x5C, 0xB6, 0xAA, 0x6E, 0x6F, 0x30,
  0x05, 0x07, 0x49, 0x40, 0x73, 0xE2, 0x14, 0xB9,
  0x82, 0xAE, 0x5B, 0xC0, 0x00, 0x58, 0x8B, 0x25,
  0xFB, 0x1D, 0x86, 0x40, 0x50, 0x3A, 0x6B, 0x3C,
  0x82, 0x0C, 0x2F, 0xBD, 0x4C, 0x9E, 0x81, 0xBE,
  0xDE, 0x5B, 0x91, 0xBF, 0xAE, 0x21, 0x98, 0xBC,
  0xF7, 0x9D, 0x1B, 0xC1, 0x21, 0x09, 0x07, 0xBB,
  0x5F, 0xA8, 0x79, 0x46, 0x84, 0x01, 0x00, 0x08,
  0x03, 0x1F, 0x19, 0x85, 0x00, 0xA4, 0x48, 0xA1,
  0xBB, 0x5F, 0xC3, 0x19, 0x3D, 0x05, 0x39, 0x82,
  0xBE, 0x6A, 0x88
};

float latitude;
float longitude;
float altitude;

void setup() {
  Serial.begin(9600);
  Serial.println(F("32 bit float"));

  uint64_t x;
  
  memcpy(&x, &data[44], sizeof(x));
  //upper 23 bits of fraction, lower 7 bits of exponent, sign bits of exponent and float
  x = ((x >> 29) & 0x000000003FFFFFFF) | ((x >> 32) & 0x00000000C0000000);
  //copy lower 32 bits of uint64_t into float
  memcpy(&latitude, &x, 4);
  
  memcpy(&x, &data[52], sizeof(x));
  x = ((x >> 29) & 0x000000003FFFFFFF) | ((x >> 32) & 0x00000000C0000000);
  memcpy(&longitude, &x, 4);
  
  memcpy(&x, &data[60], sizeof(x));
  x = ((x >> 29) & 0x000000003FFFFFFF) | ((x >> 32) & 0x00000000C0000000);
  memcpy(&altitude, &x, 4);
  
  Serial.print(F("latitude: "));
  Serial.println(latitude, 5);
  Serial.print(F("longitude: "));
  Serial.println(longitude, 5);
  Serial.print(F("altitude: "));
  Serial.println(altitude, 5);
  Serial.println();
}

void loop() {
}

Thanks for that, good to understand this a little bit more.

No worries regarding the location, I was aware when I was putting it out here. It's just random test data with no significance.

I did check out the float.exposed link that you provided. Thanks for all the help! I'm currently brushing up on my knowledge of bits, bytes and the likes - I had one flight instrumentation course in college a little over a decade ago where we went over the basics and I came out thinking I was never going to be doing this stuff ever...