Sending structs through serial.

Hi

I am currently working on a project of making a wireless universal controller with an old ps2 controller. Success so far with sending single data.

Eg
serial.write an hex value if I press a button, which is read wirelessly through Bluetooth on another board

The problem now is sending multiple commands and data at once like
-the analog readings of a particular button
-read states of multiple buttons at the same time. and so on.

My approach was using structure, (functions used in this code is not the real thing) :

struct button{
int HEX_VALUE; /*each button has its own hex value that I created*/
int analog;
};
button state;

//Example of sending command 
if (up_button_is_pressed)
{
  state.HEX_VALUE=up_button_hex;
  state.analog= read.analog(up_button);
  Serial.write(state); //my problem starts here
}

//Example of receiving command after serial reading
//Using the same struct
  if (state.HEX_VALUE==up_button_hex)
{
  Serial.println("up button is pressed");
  Serial.println("up button is pressed this hard:");
  Serial.print(state.analog);
}

My problem now is that I don't know how to send and receive struct through serial. And is there an alternative way that is better than struct to send multiple command and data.

Thank you

**Just learning how to code

The idea of a struct is good. Your variable holding the struct is a pointer to the first byte of the struct and the sizeof() function will tell you how many bytes are in the struct.

So what you can do is have a byte pointer variable pointing to the address of the struct and iterate for each byte in struct and write that byte to serial. This way you dump the memory structure in binary onto the serial port. On the receiving side you read it the same way and populate the bytes of a similarly declared structure. (you can use a union to make such code looks better than arbitrarily casting pointers)

The only thing to worry about is if your receiving computer is not same type as the sender like a UNO sending to a DUE because data types such as int would not have the same number of bytes on both sides. In that case you need to be smart and know exactly what you send and rebuild "manually" the new representation on the receiving end based on what formats should look like.

You could use a union to map your struct to a byte array and then send, or receive the array.

See this Python Binary Data demo.

...R

So what you can do is have a byte pointer variable pointing to the address of the struct and iterate for each byte in struct and write that byte to serial. This way you dump the memory structure in binary onto the serial port. On the receiving side you read it the same way and populate the bytes of a similarly declared structure. (you can use a union to make such code looks better than arbitrarily casting pointers).

Can u please explain how this can be done.

nolngrhumn:
So what you can do is have a byte pointer variable pointing to the address of the struct and iterate for each byte in struct and write that byte to serial. This way you dump the memory structure in binary onto the serial port. On the receiving side you read it the same way and populate the bytes of a similarly declared structure. (you can use a union to make such code looks better than arbitrarily casting pointers).

Can u please explain how this can be done.

You should really start a new thread, with details of your specific problem, instead of zombifying an old thread.

nolngrhumn:
So what you can do is have a byte pointer variable pointing to the address of the struct and iterate for each byte in struct and write that byte to serial. This way you dump the memory structure in binary onto the serial port.

this will dump the data as explained

struct __attribute__ ((packed)) t_demo   {
  uint16_t twoBytes;
  uint8_t  oneByte;
  uint32_t fourBytes;
};

t_demo myStruct = {0xBEBA, 0xD0, 0xEFBEADDE};

void setup()
{
  Serial.begin(115200);
  uint8_t* structPtr = (uint8_t*) &myStruct;
  for (byte i = 0; i < sizeof(t_demo); i++)  Serial.println(*structPtr++, HEX);
}

void loop() {}

you'll see it in the Serial console if you open it at 115200 bauds

on the receiving end, you just iterate and store at increasing address when you get the data

Zadiq:
My problem now is that I don't know how to send and receive struct through serial. And is there an alternative way that is better than struct to send multiple command and data.

J-M-L:
The idea of a struct is good. Your variable holding the struct is a pointer to the first byte of the struct and the sizeof() function will tell you how many bytes are in the struct.

So what you can do is have a byte pointer variable pointing to the address of the struct and iterate for each byte in struct and write that byte to serial. This way you dump the memory structure in binary onto the serial port. On the receiving side you read it the same way and populate the bytes of a similarly declared structure. (you can use a union to make such code looks better than arbitrarily casting pointers)

The following is an approximate coding translation of what has been described by @J-M-L in the above quote:
1. The given struct is:

struct button
{
   int HEX_VALUE; /*each button has its own hex value that I created*/
   int analog;
};

button state;

2. Initialize HEX_VALUE and analog with arbitrary example data:

state.HEX_VALU = 0x4567;
state.analog = 0x1234;

3. During compilation process, an array is created in RAM memory to hold the values of variables. The following codes could be executed to know the beginning address of the array and the number of bytes present in the array.

byte *ptr;     //ptr is a pointer variables 
ptr = (byte*)&state;   //ptr hold the address of the beginning location of a byte organized array

byte counter = sizeof(state);    //counter holds how many bytes the array contains (here: 4)

4. Now, execute the following codes to transfer all the bytes of the array using soft UART Port.

do
{
   byte m = (byte*)*ptr;   //read the 1st byte (it is 0x67 and not [edit]0x45) and the next byte and the next byte...
   mySerial.write(m);
   ptr++;                        //point to the location of the next byte of array
   counter--;                   //1-byte is read and transmitted
}
while(counter !=0);        //continue until all the bytes of array are read and transmitted

3. Combine the above codes to create sketch.
NANO is transmitting your struct button{}; with values for the members using soft UART Port; MEGA receives those over UART2 Port and reconstructs the structure.

MEGA Receiver:

struct button
{
  int HEX_VALUE;
  int analog;
};
button state;

byte myData[20];
bool flag = false;

void setup()
{
  Serial.begin(9600);
  Serial2.begin(9600);
}

void loop()
{
  byte n = Serial2.available();
  if (n != 0 )
  {
    if (flag == false)
    {
      char x = Serial2.read();
      if (x == '<')
      {
        flag =  true;   //struct data is coming
      }
    }
    else
    {
      Serial2.readBytesUntil('>', myData, 20);
      //--- recreate structure-------
      state.HEX_VALUE = (myData[1]<<8)|myData[0];
      state.analog = (myData[3]<<8)|myData[2];
      Serial.println(state.HEX_VALUE, HEX);
      Serial.println(state.analog, HEX);
      flag = false;
    }
  }
}

NANO Transmitting a struct data item:

#include<SoftwareSerial.h>
SoftwareSerial mySerial(3, 4);  //SRX = 3, STX = 4

struct button
{
  int HEX_VALUE;
  int analog;
};

button state;

void setup()
{
  Serial.begin(9600);
  mySerial.begin(9600);
  state.HEX_VALUE = 0x4567;
  state.analog = 0x1234;
}

void loop()
{
  byte *ptr;
  ptr = (byte*)&state;
  byte counter = sizeof(state);
  mySerial.print('<');   //beginning of struct data
  do
  {
    byte m = (byte*)*ptr;
    mySerial.write(m);
    Serial.print(m, HEX);
    ptr++;
    counter--;
  }
  while(counter != 0);
  mySerial.write('>');   //end of strauct data
  Serial.println();
  delay(2000);
}

Transmitted Data Bytes of struct:
smNano1.png
Figure-1:

Reconstructed Data Bytes of struct:
smMega1.png
Figure-2:

smNano1.png

smMega1.png

1 Like

GolamMostafa:
The following is an approximate coding translation of what has been described by @J-M-L

approximative indeed :slight_smile:

are you sure about this cast?   byte m = (byte*)*ptr;  //read the 1st byte (it is 0x45) and the next byte and the next byte...and what about little endian ?

and this will be bad      Serial2.readBytesUntil('>', myData, 20);if your data has a byte with 0x3E (and issues as well with 0x3C)

try sending and see if that works

  state.HEX_VALUE = 0x3C3E;
  state.analog = 0x3C3E;

J-M-L:
are you sure about this cast?

  byte m = (byte*)*ptr;   //read the 1st byte (it is 0x67 and not [edit]0x45) and the next byte and the next byte...

and what about little endian ?

Sorry! The comment is a mistake; it should be 0x67 (the little endian)

and this will be weird

if (x == '<')

if by any chance your data has a byte with 0x3C value

or this will be even worse

      Serial2.readBytesUntil('>', myData, 20);

if your data has a byte with 0x3E

I recognize your concerns. These are my demonstrative codes for the OP to show how the transmitted data could be reconstructed back to struct data.

To create robust sketch, we may send ASCII codes for the data bytes of the struct and then use control codes beyond 0x30 - 0x39 and 0x41-0x46 as START and END marks for a transmission frame and also CHKSUM could be used for further robustness.

GolamMostafa:
Sorry! The comment is a mistake; it should be 0x67 (the little endian)

and the cast is wrong too (should be byte not byte*)

as illustrated in my code above, to send the data just do:

  uint8_t* structPtr = (uint8_t*) &myStruct;
  for (byte i = 0; i < sizeof(t_demo); i++)  Serial.println(*structPtr++, HEX);

and that will be sent in ASCII Hexadecimal representation (each separated by CR/LF) or if you want to send the raw data

  uint8_t* structPtr = (uint8_t*) &myStruct;
  for (byte i = 0; i < sizeof(t_demo); i++)  Serial.write(*structPtr++);

no need for more

You can use SerialTransfer.h to automatically packetize and parse your data for inter-Arduino communication without the headace. The library is installable through the Arduino IDE and includes many examples.

Here are the library's features:

This library:

  • can be downloaded via the Arduino IDE's Libraries Manager (search "SerialTransfer.h")
  • works with "software-serial" libraries
  • is non blocking
  • uses packet delimiters
  • uses consistent overhead byte stuffing
  • uses CRC-8 (Polynomial 0x9B with lookup table)
  • allows the use of dynamically sized packets (packets can have payload lengths anywhere from 1 to 255 bytes)
  • can transfer bytes, ints, floats, and even structs!!

Example TX Arduino Sketch:

#include "SerialTransfer.h"


SerialTransfer myTransfer;


struct button {
int HEX_VALUE; /*each button has its own hex value that I created*/
int analog;
} state;


void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);
}

void loop()
{
  state.HEX_VALUE = 300;
  state.analog = 1000;

  myTransfer.txObj(state, sizeof(state));
}

Example RX Arduino Sketch:

#include "SerialTransfer.h"


SerialTransfer myTransfer;


struct button {
int HEX_VALUE; /*each button has its own hex value that I created*/
int analog;
} state;


void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);
}

void loop()
{
  if(myTransfer.available())
  {
    myTransfer.rxObj(state, sizeof(state));
    
    Serial.println("New Data:");
    Serial.print("HEX_VALUE: "); Serial.println(state.HEX_VALUE);
    Serial.print("analog: "); Serial.println(state.analog);
    Serial.println();
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }
}

J-M-L:
and the cast is wrong too (should be byte not byte*)

I cast by byte or byte* or no cast, the result is the same. Do they really matter -- if so, in what way?

as illustrated in my code above, to send the data just do:

  uint8_t* structPtr = (uint8_t*) &myStruct;

for (byte i = 0; i < sizeof(t_demo); i++)  Serial.println(*structPtr++, HEX);


and that will be sent in ASCII Hexadecimal representation (each separated by CR/LF) or if you want to send the raw data

uint8_t* structPtr = (uint8_t*) &myStruct;
  for (byte i = 0; i < sizeof(t_demo); i++)  Serial.write(*structPtr++);

Your codes are compact and these will serve as another good source of codes for the OP and readers of the Forum. (+K).

GolamMostafa:
I cast by byte or byte* or no cast, the result is the same. Do they really matter -- if so, in what way?

well no cast is similar to casting to byte because you declared ptr as a pointer to a byte  byte *ptr;so when you access what is pointed at, the compiler knows it's a byte.

if you cast this byte then to a (byte*) you ask the compiler to consider this single byte as an address in memory, so it will make it two bytes (as adresses on standard arduino fits on 2 bytes) and then because you store it into a byte anyway it will just dismiss the MSB. of course the optimizer gets rid of all the crap and ignore your cast :).. but it's not because it's working magically that it makes sense :), semantically what was returned was not a pointer and there is no reason to tell the compiler so...

so the right way to do it is not cast at all, you have already given all the information needed to the compiler when you declared the pointer.

My code was to spell it out what a pointer was, but that's not how I would do it... there is an even more compact way to send the information in binary format using write by simply doing:

Serial.write((uint8_t*) &myStruct, sizeof(t_demo));

J-M-L:
if you cast this byte then to a (byte*) you ask the compiler to consider this single byte as an address in memory, so it will make it two bytes (as adresses on standard arduino fits on 2 bytes) and then because you store it into a byte anyway it will just dismiss the MSB. of course the optimizer gets rid of all the crap and ignore your cast :).. but it's not because it's working magically that it makes sense :), semantically what was returned was not a pointer and there is no reason to tell the compiler so...

Cool!