I'm looking for the StreamSend.h library for use in Arduino/ESP8266 serial communications using struct

I've come across an interesting yet old article within StackExchange which has great examples of sending structured variables over serial communications. The library StreamSend.h and it's source code no longer exist on this site and I'm finding it difficult to track down. The examples shown look good and I'd like to use them in my application. My application is communication of large amounts of data between an ESP8266 and a Arduino UNO (both are combined on the same PCBA) via UART serial.

This is the article I'm referring to:

Sending Large Amounts of Serial Data

Can somebody send me the StreamSend.h library please or alternative methods for sending large amounts of data over serial (by large amounts of data I mean reading/writing say 10 to 20 variables of various data types so struct would suit well). Thank you.

Regards,

Brian

The simplest way is to use a number of Serial.print (one for each variable) to send the variables as text and use e.g. a comma as the separator. You can have a look at Serial Input Basics - updated for the receiving side and write a matching function for the sending side.

The sending of binary data is simple; cast to a byte pointer and use serial.write. Below demonstrates

struct MYDATA
{
  char text[10];
  byte aByte;
};

MYDATA dataToSend =
{
  "abc",
  0x30  // 
};

void setup()
{
  Serial.begin(57600);
  Serial.write((byte*)&dataToSend, sizeof(dataToSend));
}

void loop()
{
}

(byte*) is the cast, & indicates the address of the variable dataToSend.

There are some pitfalls

  1. Sending binary data of variables like int has a dependency on the the endianness of the two systems. Sending text is easier from that perspective.
  2. Synchronisation of binary data is more complex than of text. Reason is that it's more difficult to differentiate between actual data and a 'beginning fo packet'.

Be aware there can be issues sending a structure, as an array of bytes, from one type of Arduino to another, it may not work as you expect;

it's common practice to send a packed structure across as a binary. TLVs are one approach for delimiting different messages. for arduino, the type and length fields are read first (i.e. read 2 bytes) and then depending on the type, read the remainder of the message of length "L" directly into a structure.

Check out:

You can install it from the Arduino IDE's library manager.

Thanks gfvalvo, I installed and gave it a quick test last night but didn't seem to work. I'm testing for 'data' i.e. transferring struct from an ESP8266 to an Arduino UNO (also tested on Mega). Both processors exist on the same board i.e. dual processors. They communicate via a UART. Do you know if this library works for the ESP8266? Regards, Brian

"didn't work" is not a very useful description to assist in debugging.
Have you checked the struct's elements size and packing between the ESP8266 (32-bit architecture) and the Uno/Mega (8-bit AVR architecture)?

Thanks gfvalvo. No I did not check element size. I'll review tonight and advise. Regards, Brian

The packing and total struct size on the two different architectures will also be critical.

Thanks mate. I have written a multiplexer type code a while back for sending and receiving integers over the serial port and this worked well. Lots of breaking up words into bytes and nibbles and streaming through the buffer then rebuilding on the other side. Vice-versa to get data back. >800 lines of code so it was not code efficient (looks neat though). This was between the ESP/Arduino. I've mentioned this to give an idea of what I've done to date. I have tried my own attempt at streaming struct but had data received at the other end not making any sense so I think you're onto something with the struct size between processors. It would put logic to what I was seeing. I'll give it a go tonight. Cheers and thanks for your help so far. I'll put more effort into recording what I'm doing and will share details on the board that I'm using. I bought it from Jaycar Electronics in Australia, their part number for ESP/UNO is XC4411

https://www.jaycar.com.au/uno-with-wi-fi/p/XC4411?pos=1&queryId=090df623e927d19147cb5558cd383702

and also the ESP/Mega version

https://www.jaycar.com.au/mega-with-wi-fi/p/XC4421?pos=1&queryId=8da8bbc657cb066b86b5b6ba35b2121a

gfvalvo, I came across another post where you had made comment. Within this post I came across the attribute below which I added to my code and it work well. I'm very happy.

struct SensorData { ... } attribute((packed));

So this has fixed the problem from getting data from the ESP8266 to the UNO/Mega. Tomorrow night I will try the opposite i.e. from the UNO/Mega to the ESP8266. I don't expect any issues. What is your opinion? Do I still need to pack. I think not. Cheers and many thanks to your tips as they've sent me in the right direction. Regards, Brian

Hello gfvalvo,

Sending and receiving across the uart between the ESP8266 and the UNO appears to be working well....mostly. I get some 'air' gaps.

Overview
My aim is to setup simple reliable gateways for a set of variables to be sent and received at both ends between an ESP and UNO or Mega. Bi-directional. This will become generic code that I would apply to any Wifi ESP/UNO or Wifi ESP/Mega project with very little or no modifications. So, part of this process was to determine the maximum number of variables I could send and receive.

My code follows. I then explain the two issues:

See my code:

ESP8266 Code:

//ESP uart send and receive to Arduino UNO and Mega. For Jaycar Wifi XC4411 and XC4421 communications.

#include "SerialTransfer.h"

SerialTransfer myTransfer;

struct STRUCT {
char character1;
float variable1;
float variable2;
float variable3;
float variable4;
float variable5;
float variable6;
int variable7;
int variable8;
int variable9; //this appears to be the limit.
//int variable10;
//int variable11;
//int variable12;
//int variable13;
//int variable14;
//int variable15;
//int variable16;

} attribute((packed)); //this packing aligns the bytes as there is a difference between ESP8266 and UNO/Mega it gets out of alignment or something like that. 3 verses 4 bytes.
struct STRUCT sendStruct;

struct STRUCT receiveStruct;

void setup()
{
Serial.begin(115200);
//Serial1.begin(115200);
myTransfer.begin(Serial);

//sendStruct.character1 = '$';

}

void loop()
{
sendStruct.variable1=4.21;
sendStruct.variable6=7.43;
sendStruct.variable7=1234;
sendStruct.variable8=5678;
myTransfer.sendDatum(sendStruct);
delay(500);

if(myTransfer.available())
{
myTransfer.rxObj(receiveStruct);
}

sendStruct.variable2=receiveStruct.variable1; //loop back test

}

UNO Code:

//Arduino UNO and Mega uart send and receive to ESP. For Jaycar Wifi XC4411 and XC4421 communications.

#include <LiquidCrystal.h>
//pin defs to suit LCD Shield
LiquidCrystal lcd(8, 9, 4, 5, 6, 7);

#include "SerialTransfer.h"

SerialTransfer myTransfer;

struct STRUCT {
char character1;
float variable1;
float variable2;
float variable3;
float variable4;
float variable5;
float variable6;
int variable7;
int variable8;
int variable9; //appears to be the limit
//int variable10;
//int variable11;
//int variable12;
//int variable13;
//int variable14;
//int variable15;
//int variable16;
} attribute((packed)); //this packing aligns the bytes as there is a difference between ESP8266 and UNO/Mega it gets out of alignment or something like that. 3 verses 4 bytes.
struct STRUCT receiveStruct;

struct STRUCT sendStruct;

void setup()
{
Serial.begin(115200);
//Serial1.begin(115200);
myTransfer.begin(Serial);

lcd.begin(16, 2);
lcd.setCursor(0, 1);
//lcd.print(" Test");

}

void loop()
{
myTransfer.sendDatum(sendStruct);
delay(500);

if(myTransfer.available())
{
myTransfer.rxObj(receiveStruct);

lcd.setCursor(1, 0);
lcd.print(receiveStruct.variable1);
lcd.setCursor(1, 1);
lcd.print(receiveStruct.variable2);
lcd.setCursor(7, 0);
lcd.print(receiveStruct.variable3);
lcd.setCursor(7, 1);
lcd.print(receiveStruct.variable6);
lcd.setCursor(12, 0);
lcd.print(receiveStruct.variable7); //there is some kind of alignment issue. int variables don't line up. Every second one is empty.
//lcd.setCursor(12, 1);
//lcd.print("    ");
lcd.setCursor(12, 1);
lcd.print(receiveStruct.variable9); //variable9 aligns with variable8 in ESP

}

sendStruct.variable1 = receiveStruct.variable2+0.01; //loop back test

}

The issues:
Issue 1: Found a limit of around ten variables. If I increased further the data transfer appeared to stop.

Issue 2: A miss alignment of variables. In the code above I use a two line LCD 6 character display to show the data being transferred. I do a simple loop back test to prove data is going both ways. I do this as with the hardware config I need to set up DIP switches so the ESP and UNO can talk. Once this is done I can't use the serial monitor function in the IDE. So, I've noted when sending variables sendStruct.variable7 and sendStruct.variable8 then only sendStruct.variable7 aligns to the
receiveStruct.variable7 but receiveStruct.variable8 is empty and receiveStruct.variable9 has the data.

Questions

  1. What do you think the limit of struct data would be? I.e. how many variables can I expect to stream?
  2. What do you think is happening with the alignment? Note I did not appear to have an alignment issue with the floats (tested only 6). I have not checked what would happen if I changed the struct to only integers or put the integers first before the floats within the struct.

Any pointers would be appreciated.

Best regards,
Brian

Hello All,

Just to wrap up this Forum post, I resolved my issues by simply using float variables and converting data once it was streamed thought the serial uart. I've noted on other posts there's a bit of adjustment to be done if packing the data is required. Looked ok but too much work for me. Floats are fine between ESP and UNO/Mega.

Regards,

Brian