Rebuild Structure sent through Serial

Hi,

I've been fighting for 2 days on something I don't understand and finally decided to ask for some help.
I'm using 2 MEGA here.

The code is very long so I'll try to explain instead of just pasting everything.

I'm trying to transfer multiple structure from one MEGA to Another using easytransfer library but due to some limitations (which I assume may come from my lack of skills), I decided to work the following way :

On both arduino, I created 2 structures each containing a buffer ; easytranfer will do its job and synch the structure.

  • Arduino 1 : send buffer of 50 char // receive buffer of 50 char
  • Arduino 2 : receive buffer of 50 char // send buffer of 50 char

This part is working as intended, I can write data in send buffer, find it in receive buffer and do what I want with it.

But now I want to copy a structure in the buffer, let easytransfer synch it for me and rebuild the structure on the other side.

All structures I need to transfer exists on both sides and are declared the exact same way. There is no compil error.

Here is the important part of the structure (same code on both arduino):

.
.
.

typedef struct GLOBAL{
  unsigned long magic;
  uint8_t struct_version;
  uint16_t checksum;
  char profile_name[9];
  struct NETWORK_CONFIG network;
  struct PLATFORM_CONFIG platform;
  struct SERVO_CONFIG servo[8];
}config;

typedef struct NETWORK_CONFIG{
  byte mac[6];
  IPAddress ipV4;
  uint16_t port;
  IPAddress gateway; 
  IPAddress netmask; 
  IPAddress in_ipV4_min;
  IPAddress in_ipV4_max;
};

typedef struct SYNCHRONIZED_BUFFER{
  char buffer[50];
};

config active_profile;
config test_profile; //this one is just to play with data and is used as a proof of concept later in the code.
SYNCHRONIZED_BUFFER synchronized_buffer_rcv;
SYNCHRONIZED_BUFFER synchronized_buffer_send;
.
.
.

On arduino 1, the value of active_profile.network.port is 8704. In order to play around and understand how it works I made this code which will prepare a buffer with a set of command followed by the structure.

      char s[sizeof(active_profile.network)];
      char c[50]="md st n ";
      memcpy(s, &active_profile.network, sizeof(active_profile.network));
      memcpy(c+8, s, sizeof(s));
      send_command_to_serial(c); //will transfer c to the receive buffer of the other arduino.
      
//code below is to make sure it's working as I think it should...
      memcpy(&test_profile.network, c+8, sizeof(test_profile.network));
      Serial.println("Port : ");
      Serial.println(test_profile.network.port);

//test_profile.network.port = 8704 so test is OK for me, typicaly I just copied the data from a char array buy ignoring part of the array containg what i would call the "command". 

      Serial.println(c); //It returns this : md st n Þ­¾ïþïK[03]À¨
      //i'm now expecting to find the exact same thing in the buffer on the second arduino.

Now on the second arduino, I confirmed that the receive buffer is equal to md st n Þ­¾ïþïK[03]À¨ so basicaly, exactly what I want.

Still on Arduino 2, I'm using the following code to take the structure data from the receive buffer into the test_profile :

      memcpy(&test_profile.network, synchronized_buffer_rcv.buffer+8, sizeof(test_profile.network)); //the +8 is to start reading data structure after "md st n " 
      Serial.println("\nPort : ");
      Serial.print(test_profile.network.port);

This is where i'm lost. As the memcpy worked on first arduino, i'm expecting to see it work on the second one. But test_profile.network.port = 0 instead of 8704 no matter what I try and no matter the fact that send buffer on one side is :

  • md st n Þ­¾ïþïK[03]À¨

and receive buffer on the over side is :

  • md st n Þ­¾ïþïK[03]À¨

It looks the same to me so i'm definitely missing something out.

Serial.println(c); //It returns this : md st n Þ­¾ïþïK[03]À¨

You need to print out the individual characters as hex values. Printing c as a C-string will only print to the first 0 byte and so may hide some of the content. (Won't solve your problem though.)

Edit: And how is send_command_to_serial(c) implemented? Does it stop at a 0 byte too?

If you want to send the whole struct at once, make a union with it and a byte buffer and then you don't have to convert anything. Especially since both ends are Arduino and you can be relatively certain that they both put the struct together the same way in memory.

On the sending end you would just send out the byte buffer view of the union. On the receive side you would read bytes in one by one into the byte buffer view of the union and on the very next line you can start treating it like the struct and using it as an object instead of a buffer full of bytes.

Something along these lines.

This is untested, uncompiled, not even really proof-read that well. Just trying to illustrate the point.

union Send-A-Struct  {
      myStruct asStruct;
      byte asBuffer[sizeof(MyStruct);
}

Send-A-Struct sender;
sender.asStruct = instanceOfMyStruct;

Serial.write(sender.asBuffer, sizeof(sender.asBuffer));
Send-A-Struct receiver;

Serial.readBytes(receiver.asBuffer, sizeof(receiver.asBuffer));

anotherInstace = receiver.asStruct;

Serial1.print(anotherInstance.someMember);  // and someMember will have the new received value

If you want to get tricky, you could even wrap all the member variables in a union in the struct definition itself and then you wouldn't have to create and copy them at all.

You don't have to copy the data at all if you just typecast it.

You don't even need to use a union.

If you use the name of the struct as an address you can just write the values to successive addresses after that - something like this pseudo code

myStructAddr = &myStruct
newByte = Serial.read()
(myStructAddr + n) = newByte
n++

I'm not sure if I have the reference to myStruct correct and I can't immediately find the code where I did this.

...R

Hi again !

Thanks a lot for all your reply. I'll test as soon as i'm back home later today :slight_smile:

arduarn:

Serial.println(c); //It returns this : md st n Þ­¾ïþïK[03]À¨

You need to print out the individual characters as hex values. Printing c as a C-string will only print to the first 0 byte and so may hide some of the content. (Won't solve your problem though.)

Edit: And how is send_command_to_serial(c) implemented? Does it stop at a 0 byte too?

Understood. So I will display the hex content of the buffer on both side and compare. If the problem comes from that, i'll definitely find a difference. That will be the first step to explain why it didn't work.

You asked me about the send_command_to_serial.

send_command_to_serial(char command[50]){
 memset(synchronized_buffer_send.buffer, '\0', sizeof synchronized_buffer_send.buffer);
 strcpy(synchronized_buffer_send.buffer,command);
 ET_buffer_send.sendData(); \\this part is from the easytransfer library. 
}

As far as I understand, the easytransfer library will replicate the buffer to the receive buffer on the other arduino, no matter the content and run a checksum to ensure that send and receive buffer content are the same. I checked the library, and I don't understand the checksum part... It looks like it's only counting the number of bytes to send and checking the number of bytes received on the other side... Meaning that the whole structure may be corrupted.

I'm using a checksum (real one) when reading data from EEPROM so I may implement something similar here.

Delta_G:
If you want to send the whole struct at once, make a union with it and a byte buffer and then you don't have to convert anything. Especially since both ends are Arduino and you can be relatively certain that they both put the struct together the same way in memory.

On the sending end you would just send out the byte buffer view of the union. On the receive side you would read bytes in one by one into the byte buffer view of the union and on the very next line you can start treating it like the struct and using it as an object instead of a buffer full of bytes.

I didn't know UNION yet. So i've just read a tutorial here (in fr sorry) https://openclassrooms.com/courses/les-unions which proved to be very instructive. If I understand this concept correctly, all variable declared in the union are sharing the same address range and the range size is equal to the biggest "object" declared in the union. In your example, the memory range reserved would start at &asBuffer and end at &asBuffer[sizeof(MyStruct)] with &asBuffer = &asStruct. That would be indead very useful if I wanted to create my own library (which I will certainly do as a training purpose). That will be next step !

Jimmus:
You don't have to copy the data at all if you just typecast it.

Ok, noted. I understand what typecast is but I don't see yet how I can use your suggestion in my problem. Are you refering to the fact that I memcopy everything and that I could avoid doing this by using typecast ?

Robin2:
You don't even need to use a union.

If you use the name of the struct as an address you can just write the values to successive addresses after that - something like this pseudo code

myStructAddr = &myStruct

newByte = Serial.read()
(myStructAddr + n) = newByte
n++




I'm not sure if I have the reference to myStruct correct and I can't immediately find the code where I did this.

...R

Got it, will use it when I build my own transfer library (which would be next step as said earlier).

So my next action will be to compare hex value of both buffers which will supposedly justify why it's not working. Based on what I discover, I will use your suggestions but the more I read your ideas the more I think a serial transfer "protocol" is missing to exchange data between arduino.

Thanks a lot for your time and help !

strcpy(synchronized_buffer_send.buffer,command);

Yeah, well that's likely your problem then. The command isn't a string, so you can't treat it like one.

Yeah, strcpy is not the right function to use here. It will stop at the first 0 byte.

Really, though, doesn't EasyTransfer do all this for you? Wouldn't it be easier and way more efficient to just use your structures directly?

I just came back home, jumped on my arduino, edited the send_command_to_serial(char command[50]) function and replaced

strcpy(synchronized_buffer_send.buffer,command);


by 

- ```
memcpy(synchronized_buffer_send.buffer,command,50);

And it worked ! ! ! Thank you so much for your help :slight_smile: :slight_smile: ! ! ! !

Jimmus:
Yeah, strcpy is not the right function to use here. It will stop at the first 0 byte.

Really, though, doesn’t EasyTransfer do all this for you? Wouldn’t it be easier and way more efficient to just use your structures directly?

This is what i tried at first. But for a reason I may not fully understand yet, I only got the Easytransfer library to work with one send structure and one receive structure per device.

I tried using multiple structure by assigning one send and one receive on Serial1, another 2 structs on Serial2 but the result was a complete mess with the value of one struct overwritting the value of others on the received buffer… I tried with small structure of less than 20 bytes… I also tried 125 bytes structure (containing all structure) but the error rate was too high with only 1 successful transfer every few minutes …

I used a simplified sketch for test purpose and only managed to get it work with 2 structs of less than 50 bytes for a 100% success rate other 8 hours of test. This is why I tried to “workaround” the problem. And even now, I don’t know if the limits comes from the library or because i’m not using it correctly, so I’ll make sure to figure that out later :slight_smile:

Once again thanks for you help guys !