Anything wrong with this method of packing floats in to byte arrays?

Projects Programming
1

Is there anything wrong with the following way of packaging float data in to byte arrays (for sending over communication links, eeprom storage...) and recovering them afterwards?

I've tested and it does work, with no change of value to the floats (even to far more decimal places than they are stored to) ater recovery. But I don't know whether there is something about using memmove functions and pointers in this way which is frowned upon, or not guaranteed to work in all circumstances.

I know I have seen warnings against trying to do this transfer of data betwen variable types by use of union, and other pointer related methods, even though they appear to work.

My code is below, it can be compiled and run as it is, it demonstrates taking an array of floats, putting them in to a byte array, then copying that byte array in to another bunch of floats to reconstruct the originals. It then edits the originals and the byte array to prove this doesn't modify the copied floats (proving it is copy by value, not reference). And then edits the copies to show this doesn't affect the originals. It also copies out one float to be stored in a non-array float.

// the setup function runs once when you press reset or power the board
void setup() {
  // initialize digital pin LED_BUILTIN as an output.
  //pinMode(LED_BUILTIN, OUTPUT);
  Serial.begin(115200);
  Serial.println("Float to array test");

  volatile float Test1[5]={23.4,3.14159,87.1,-1.0005,0.0046};
  volatile float Test2[5]={0};
  volatile uint8_t ByteSet[20]={0};
  Serial.println("Initially");
  Serial.println("T1: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test1[i],24);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("T2: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test2[i],12);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("BS: ");
  for(uint8_t i=0; i< 20; i++){
    Serial.print((int)ByteSet[i]);
    Serial.print(", "); 
  }
  Serial.println(" ");

  for(uint8_t i=0; i< 5; i++){
    float* pf = &Test1[i];
    uint8_t* pb = &ByteSet[i*4];
    memmove(pb,pf,sizeof(Test1[i]));
  }

  Serial.println("Copied in to bytes");
  Serial.println("T1: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test1[i],12);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("T2: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test2[i],12);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("BS: ");
  for(uint8_t i=0; i< 20; i++){
    Serial.print((int)ByteSet[i]);
    Serial.print(", "); 
  }
  Serial.println(" ");

  for(uint8_t i=0; i< 5; i++){
    uint8_t* pb = &ByteSet[i*4];
    float* pf = &Test2[i];
    memmove(pf,pb,sizeof(float));
  }

  float Var3=0;
  uint8_t* pb = &ByteSet[3*4];
  float* pf = &Var3;
  memmove(pf,pb,sizeof(float));

  Serial.println("Copied back from bytes");
  Serial.println("T1: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test1[i],12);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("T2: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test2[i],24);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("BS: ");
  for(uint8_t i=0; i< 20; i++){
    Serial.print((int)ByteSet[i]);
    Serial.print(", "); 
  }
  Serial.println(" ");
  Serial.print("Var3 ");
  Serial.println(Var3,12);

  float Var33=Var3+10;



  for(uint8_t i=0; i< 5; i++){
    if(i<4){
      Test1[i]=Test1[i+1]+2.3; 
    }else{
      Test1[i]=Test1[i]-6;
    }
  }

  for(uint8_t i=0; i< 20; i++){
    ByteSet[i]=11;
  }

  Serial.println("Originals edited");
  Serial.println("T1: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test1[i],12);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("T2: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test2[i],12);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("BS: ");
  for(uint8_t i=0; i< 20; i++){
    Serial.print((int)ByteSet[i]);
    Serial.print(", "); 
  }
  Serial.println(" ");
  Serial.print("Var3 ");
  Serial.println(Var3,12);
  Serial.print("Var33 ");
  Serial.println(Var33,12);

  for(uint8_t i=0; i< 5; i++){
    Test2[i]=5.0*(Test2[i]+100.0);
  }
  Var33=2.971;
  
  Serial.println("Copies edited");
  Serial.println("T1: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test1[i],12);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("T2: ");
  for(uint8_t i=0; i< 5; i++){
    Serial.print(Test2[i],24);
    Serial.print(", "); 
  }
  Serial.println(" ");

  Serial.println("BS: ");
  for(uint8_t i=0; i< 20; i++){
    Serial.print((int)ByteSet[i]);
    Serial.print(", "); 
  }
  Serial.println(" ");
  Serial.print("Var3 ");
  Serial.println(Var3,12);
  Serial.print("Var33 ");
  Serial.println(Var33,12);
}

// the loop function runs over and over again forever
void loop() {

}

Thank you

2

What's with all that volatile?

3

Don't think you need the volatiles either..
Can also typecast the float array and send directly..

  float Test1[5]={23.4,3.14159,87.1,-1.0005,0.0046};
  float Test2[5]={0};
  uint8_t ByteSet[20]={0};

Serial.write((uint8_t*)Test1,sizeof(Test1));
memmove(ByteSet,Test1,sizeof(ByteSet));
Serial.write(ByteSet,sizeof(ByteSet));
memmove(Test2,ByteSet,sizeof(ByteSet));
Serial.write((uint8_t*)Test2,sizeof(Test2));

have fun.. ~q

4

For EEPROM storage, you can put all the floats, along with other types of variables, into a struct and use the put() / get() functions to write / read the entire struct.

5

I just typed "volatile" by reflex when writing this, I'd been doing a lot of coding recently where I wanted to test how long certain operations took by timing with an oscilloscope, and had to use volatile to make sure that pointless calaculations I inserted (for the purpose of seeing how long they take for example values) didn't get optimised away.

Post #3, I set this up to test copying individually as well as putting whole arrays of floats in the byte array at once. I wrote it the way I did so I could easily adapt it if I wanted to copy independent floats (non-arrayed) to manually defiend array places. I guess my code is as valid as yours though, just set out differently?

I've seen some warnings about "union type punning in C++" being potentially undefined, does this apply here, or is this method safe from any concerns about that? And as Endianness warnings go, endianness for floats is the same on all AVR microcontrollers?

6

The correct phrase is "causes undefined behavior". The distinction is important for differentiating between garbage and quality when searching the internet.

Causes undefined behavior. Yes it does.

Are you using a union?

Endianness for processors like the AVR processor is determined by the floating point emulation typically provided by the compiler vendor. In the Arduino world that's usually avr-gcc. In other words, as long as you're consistently using avr-gcc to build then the endianness for floating point numbers is consistent (little).

7

Casting away the volatile qualifiers and then using the non-volatile pointers to access the volatile objects in the Test1/2 and ByteSet arrays invokes Undefined Behavior.
The compiler should have warned you about that, so please go to the IDE Preferences and turn up the warning level.

See this page on type punning for conversions from float arrays to byte arrays: Don't use unions or pointer casts for type punning

8

PieterP: Without the volatiles my method is alright then?

9

I'd use memcpy instead of memmove if the arrays do not overlap, but yes.

Except if it's just for serialization, then there's no need for the memcpy at all, as explained in the “Cast to a character array”

10

How do you incorporate

alignas(float) uint8_t bytes[] {0xA4, 0x70, 0x45, 0x41};
float x = *std::start_lifetime_as<float>(bytes); // Ok

into an Arduino sketch?

11

Right now, you can't.
In a couple of months, you should be able to patch your Arduino install with the latest version of GCC, and add the -std=c++23 flag to your platform.txt file.

To be able to use it in a portable Arduino sketch, you'll have to wait for the Arduino devs to finally update their C++ standard version, but I wouldn't hold my breath, because some cores are still on a 12 year old version :frowning:

12

Thanks.
 

Failing that, what's an alternative in the DE-serialization direction?

13

Just memcpy, I'm afraid.