How to put a "RAW" float (4 byte) in a buffer , and how to reconstruct it ?

[tfoutfou]

HI ,

i am sending value over the internet with a GSM module , and data is limited so i am trying to reduce my string to the bare minimum

A float is 4 byte right ?

how to put those 4 byte in a buffer

something like that but compiler not happy about it

byte buffer[5];
float floatNumber = 1.15478973;

buffer[0] = floatNumber >> 24;
buffer[1] = floatNumber >> 16;
buffer[2] = floatNumber >> 8;
buffer[3] = floatNumber;

and how to reconstruct it as a float on the other side ?

byte buffer[5];
float floatNumber;

floatNumber = buffer[3];
floatNumber += buffer[2] << 8;
floatNumber += buffer[1] << 16;
floatNumber += buffer[0] << 24;

i have searched already but not sure about what i have to ask on google to get an answer

all the answer that i found put the value of float in the buffer and end up taking 10 byte for (1.15478973)

Thanks

[horace]

the simplest way to test an algorithm is to write a simple program
the first problem identified is

error: invalid operands of types 'float' and 'int' to binary 'operator>>'
    8 | buffer[0] = floatNumber >> 24;
      |             ~~~~~~~~~~~ ^~ ~~
      |             |              |
      |             float          int

[david_2018]

@tfoutfou

and data is limited so i am trying to reduce my string to the bare minimum

If the data needs to be in the form of a string, then you can't send the raw data in a float, because that may contain a byte with a value of 0, which will be seen as the end of the string.

You could use memcpy() to copy the float into an array, or use pointers to copy the bytes individually.

[tfoutfou]

i will be using gsm.write(buffer, my_fixed_and_already_known_length)

i have already think about the 0 in the string

not familiar with memcpy , will look at it

[horace]

try a union

union Data {
  byte bytes[4];
  float f;
} data, data2;

void setup() {
  Serial.begin(115200);
  delay(2000);
  Serial.println(sizeof(float));
  data.f = 1.15478973;
  for (int i = 0; i < 4; i++)
    { Serial.print(data.bytes[i], HEX); Serial.print(' '); }
  Serial.println();
  data2 = data;
  Serial.println(data2.f);
}

serial monitor prints

4
26 D0 93 3F 
1.15

also try memcpy() as suggested by @david_2018

be careful with doubles on a UNO they are 4 bytes on an ESP32 8 bytes

[david_2018]

Also need to be careful of the endianness of the processors at each end of the transmission. Some processors store the least significant byte of a variable at the lowest memory address (little endian), other store the most significant byte at the lowest address (big endian).

[horace]

@david_2018

Also need to be careful of the endianness of the processors at each end of the transmission.

@david_2018 makes a good point
also the number of bytes to store integer types can vary from processor to processor, e.g. int on a UNO is 16bits on an ESP32 is 32bits
if you know the size of your data use int16_t, int32_t, etc - see Fixed width integer types
even more tricky can be transmitting binary data between languages, e.g. C++ to java or Python

[tfoutfou]

endianness or data type length is not a problem , i am familiar with those

[gfvalvo]

@tfoutfou

how to put those 4 byte in a buffer

@horace

try a union

Type Punning via a union is not permitted in C++ (it is in C).

Use memcpy():

  constexpr size_t floatFize {sizeof(float)};
  byte buffer[floatFize];
  float floatNumber = 1.15478973;

  memcpy(buffer, &floatNumber, floatFize);

On the RX side:

  memcpy(&floatNumber, buffer, floatFize);

[tfoutfou]

since i may want to put my 4 byte float IN the middle of my final string and get my float FROM anywhere in the string , memcpy is absolutely the way to go

(my string is fixed length and will alway have a float at [17] for the exemple)

.

Union is too new to me to understand how it work , and i am not sure if i can do something like that with union

byte buffer[35];
float floatNumber = 1.15478;


void setup() {
 Serial.begin(115200);
 memcpy(&buffer[17], &floatNumber, 4);
 floatNumber = 0;
 memcpy(&floatNumber, &buffer[17], 4);
 Serial.println(floatNumber, 5);
}

void loop() {

}

[GoForSmoke]

You could write a few floats to a file, close it and re-open it to read bytes. Once you are sure of the format you can read a float from any address in the file, no need to play put-together or take-apart at all. The file isn’t memory space, the compiler doesn’t control it like RAM. Even the buffer is like that, you only read and write to buffers.

[J-M-L]

@tfoutfou

and will alway have a float at [17]

just to be sure we are in line there : the float will start at index 17 but will occupy bytes at 17, 18, 19 and 20.

@tfoutfou

i may want to put my 4 byte float IN the middle of my final string

not sure I really understand what you mean. If you have a text string with 34 characters (and a null terminator I assume) that is something a human can read, why would you store a binary representation in the middle of it ? You won't be able to print the text anymore...

[gfvalvo]

@J-M-L

If you have a text string with 34 characters (and a null terminator I assume) that is something a human can read) why would you store a binary representation in the middle of it ? You won't be able to print the text anymore...

Wonder what data types are in the rest of that buffer. Seems like a struct might be more appropriate here than a byte array. Probably best to do so with __attribute__((packed))

[tfoutfou]

yes i know it will occupy 4 bytes a 17 , 18 , 19 ,20

.

i call it a string , but it doesnt have to be human readable , it’s just for sending data to a remote location and use less data on my very limited IOT mobile plan

.

string will be fixed length and something like 3 start char like "ABC”

a bunch of long , int , float , that will always have the same order and length

4 byte for CRC

3 end char like “DEF”

i do that because it’s simpler for me , and with my limited knowledge , i know how to parse it and get my data at the remote side

[J-M-L]

@tfoutfou

i call it a string , but it doesnt have to be human readable , it’s just for sending data to a remote location and use less data on my very limited IOT mobile plan

OK, we usually call that a payload then.

a classic way to make your life easy is indeed a packed struct as @gfvalvo suggested

@gfvalvo

Probably best to do so with __attribute__((packed))

@tfoutfou

will be fixed length and something like 3 start char like "ABC”

a bunch of long , int , float , that will always have the same order and length

4 byte for CRC

3 end char like “DEF”

consider something like this which will be 35 bytes

struct __attribute__((packed)) Payload {
  char start[3];        // bytes 0-2 ➜ your ABC
  uint32_t field1;      // bytes 3-6
  uint32_t field2;      // bytes 7-10
 // ...
  float myFloat;        // bytes 17-20
  // ...
  uint32_t crc;         // bytes 28-31
  char end[3];          // bytes 32-34 ➜ your DEF
};

Payload aPayload;

then to fill in your float you just do aPayload.myFloat = 3.1415927;

(in practice we probably would not store the header and trailer inside each structure, they would just be added in front and at the end of the payload when you send it out)

[horace]

by keeping to even word boundaries I try to avoid attribute_((packed)) and the overhead of packing the structure at the transmitter and unpacking at the receiver
e.g. used on a LoRa client/server network with multiple clients and multiple structures

// test packet - must match receiver
struct  Struct1 {
  byte StructureID;  // identifies the structure type
  byte NodeID;        // ID of transmitting node
  int16_t seq;          // sequence number
  int16_t distance;
  float voltage;
  char text[20];
  unit16_t crc;
};
Struct1 struct1 = { structureID, nodeID, 0, 1, 4.5, "hello",0 };  // test data

the sequence number can be used to check for lost/duplicate packets and the crc as an error check
as a check I usually print the

  Serial.printf("sizeof(Struct1) %d \n", sizeof(Struct1));

at the transmitter/receiver to make sure there are no problems
however, when dealing with multiple different processors have I have used attribute_((packed))
does not help when transmitting to Java though!

[J-M-L]

PS/

in practice we probably would not store the header and trailer inside each structure, they would just be added in front and at the end of the payload when you send it out

#include <Arduino.h>

struct __attribute__((packed)) Payload {
  // ....
  float myFloat;
  // ....
  uint32_t crc; // has to be the last element 
};

// CRC32 (polynomial 0x04C11DB7)
uint32_t crc32(const uint8_t* data, size_t length) {
  uint32_t crc = 0xFFFFFFFF;
  for (size_t i = 0; i < length; i++) {
    crc ^= ((uint32_t)data[i] << 24);
    for (uint8_t j = 0; j < 8; j++) {
      if (crc & 0x80000000) crc = (crc << 1) ^ 0x04C11DB7;
      else crc <<= 1;
    }
  }
  return crc ^ 0xFFFFFFFF;
}

void send(Payload &aPayload) {

  // update crc
  aPayload.crc = crc32((uint8_t*)&aPayload, sizeof aPayload - sizeof aPayload.crc);

  Serial.write("ABC", 3); // send header
  Serial.write((uint8_t*)&aPayload, sizeof(aPayload)); // send payload and CRC
  Serial.write("DEF", 3); // send trailer 
}

[J-M-L]

@horace

by keeping to even word boundaries I try to avoid attribute_((packed)) and the overhead of packing the structure at the transmitter and unpacking at the receiver

On a 32-bit system, aligning data on even addresses (2-byte boundaries) is not enough because the CPU usually requires natural alignment, which for 32-bit types means 4-byte boundaries.

Also if you have different MCU on the sender and receiver, you might end up with a different alignment if you don't pack.

what do you mean by "packing the structure at the transmitter and unpacking at the receiver". if fields are not aligned you pay indeed the cost of accessing across natural boundaries but there is nothing to pack and unpack ?

[kenb4]

Can do static_assert with offsetof to check that the packed worked as expected (which it did on ESP32)

struct __attribute__((packed)) Payload {
  char start[3];        // bytes 0-2 ➜ your ABC
  uint32_t field1;      // bytes 3-6
  uint32_t field2;      // bytes 7-10
  char filler1[6];
  float myFloat;        // comment here: bytes 17-20
  char filler2[7];
  uint32_t crc;         // bytes 28-31
  char end[3];          // bytes 32-34 ➜ your DEF
};
static_assert(sizeof(Payload) == 35);
static_assert(offsetof(Payload, myFloat) == 17);
static_assert(offsetof(Payload, end) == 32);

void setup() {
  Serial.begin(115200);
}

void loop() {
  static Payload p;
  static uint64_t i;
  ++p.myFloat;  // operation works as expected -- for a while, anyway
  if (++i % 100000 == 0) {
    Serial.print(i);
    Serial.print('\t');
    Serial.println(p.myFloat);
  }
}

Note that the code assistance on hover will include the offset

field myFloat

---

Type: float
Offset: 17 bytes
Size: 4 bytes
comment here: bytes 17-20

---

// In Payload
public: float myFloat

[oldcurmudgeon]

@J-M-L

what do you mean by "packing the structure at the transmitter and unpacking at the receiver". if fields are not aligned you pay indeed the cost of accessing across natural boundaries but there is nothing to pack and unpack ?

The user doesn't have to pack and unpack, but as you noted there is a cost. See toward the end of my first post in the rant here to see what the compiler has to do to "fix" it.

Why I hate __attribute__((packed))

Once again I have seen posts promoting use of __attribute__((packed)) on a struct. I can't stand it anymore, so I have written this rambling post on why I think that is a bad solution. The packed attribute is commonly used when a poorly designed structure is shared between processors with different bus widths. Lets see how this could happen. Say you are building a small system where you have a couple of UNOs talking to a MEGA 2560. You build a struct to hold the data something like this. You …