[Resolved] Bidirectional Serial Uno <=> Esp8266

Hello :slight_smile:

First of all first of all thank you for all the help I have been able to find here so far in looking for answers. This is the first time I have to ask my own question, because my limited knowledge at the lower level is blocking me...

I try to exchange data between my Adafruit HUZZAH which contains an ESP8266 and my Arduino Uno (because I need an Arduino to control things like LED stripes).

I succeed with basic tests in both ways but it became more complicated when I wanted to do it simultaneously. Indeed, I can receive real time data from both boards and I read somewhere (?) the HUZZAH can only received or send data at the same time (and I saw corrupted data in my serial display when doing that).

I spare you my many attempts but finally I tried to used EspSoftwareSerial to replace the hardware Serial because its seems to be able to manage that :

Except at high bitrates, depending on other ongoing activity, interrupts in particular, this software serial adapter supports full duplex receive and send.

And it seems to work but, as suggested on the library page, it could generate errors. I use a very low baud rate but I have about 5% of characters that appear corrupted. My knowledge in such domain are weak, I thought about sending several times the data, use checksum etc... Finally I tried to use Reed-Solomon Forward Error Correction Library. It's better but not perfect (still 2-3% corruption).

As I've tried a lot of things, I end up wondering if I'm doing things right!

What this testing code do is quiet simple:

  • When Huzzah is connected to the wifi, it sends "<TOPIC?>"
  • When Uno receives "<TOPIC?>", it sends several packages with "<TOPIC!mqtt_topic>"
  • At each received "<TOPIC!mqtt_topic>", Huzzah sends (right away) the reformatted command
  • The Uno displays the received command

The 3rd line is done for testing purpose. I suppose it can be done after all the commands is received from Uno but I suppose that sometimes, data will be transferred both ways at the same time so that why I keep it that way to try to resolve the issue. Maybe I'm not clear and maybe it makes no sense ^^'

Uno Code

#include <RS-FEC.h>
#include <SoftwareSerial.h>

#define MAX_STRING_LEN 64

SoftwareSerial mySerial(10, 11); // RX, TX
String fromMQTT;
byte index=0;

const char SOPmarker = '<'; //This is the start of packet marker
const char EOPmarker = '>'; //This is the end of packet marker
char serialbuf[64]; //This gives the incoming serial some room. Change it if you want

char *MQTT_topics[] = {"led/state", 
                      "switch/state", 
                      "switch/connected"};

const int msglen = 60;  const uint8_t ECC_LENGTH = 10;  //Max message lenght, and "guardian bytes", Max corrected bytes ECC_LENGTH/2
char message_frame[msglen]; // The message size would be different, so need a container
char repaired[msglen];    char encoded[msglen + ECC_LENGTH];

RS::ReedSolomon<msglen, ECC_LENGTH> rs;
                      
void setup() {
  
  Serial.begin(115200);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }
  Serial.println("Running on the Arduino");
 
  pinMode(13,OUTPUT);
 
  // set the data rate for the SoftwareSerial port
  mySerial.begin(4800);
}

void loop() {
  
  if (mySerial.available()) {
    
    static int bufpos = 0; 
    char inchar = mySerial.read();
    
    if (inchar != EOPmarker && inchar != SOPmarker) { 
      serialbuf[bufpos] = inchar; 
      bufpos++;
      
    }else if (inchar == SOPmarker) { 
      bufpos = 0;
    
    }else{
      
      serialbuf[bufpos] = 0; 
      bufpos = 0;

      Serial.print("received: ");
      Serial.println(serialbuf);

      if(strcmp(serialbuf, "TOPICS?") == 0) {

        for (int i = 0; i < sizeof(MQTT_topics[i]); i++) {
          char message[] = "<TOPIC!";
          strcat(message,MQTT_topics[i]);
          strcat(message,">");
          
          memset(message_frame, 0, sizeof(message_frame));        // Clear the array
          for(int i = 0; i <= msglen; i++) {    message_frame[i] = message[i];     } // Fill with the message
         
          rs.Encode(message_frame, encoded); 
          mySerial.print(encoded);
        }
      }
    }
  }
  
 }

char* subStr (char* input_string, char *separator, int segment_number) {
  char *act, *sub, *ptr;
  static char copy[MAX_STRING_LEN];
  int i;
  strcpy(copy, input_string);
  for (i = 1, act = copy; i <= segment_number; i++, act = NULL) {
    sub = strtok_r(act, separator, &ptr);
    if (sub == NULL) break;
  }
  return sub;
}

Huzzah Code

#include <RS-FEC.h>
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

#include <SoftwareSerial.h>

#define BAUD_RATE 4800
#define MAX_STRING_LEN 64

#define LED_PIN 5
#define BUTTON_PIN 4

bool debug = false; 

String fromUno;
const char SOPmarker = '<'; //This is the start of packet marker
const char EOPmarker = '>'; //This is the end of packet marker
char serialbuf[64]; 

// From Arduino-FEC example
const int msglen = 60;  const uint8_t ECC_LENGTH = 10;  //Max message lenght, and "guardian bytes", Max corrected bytes ECC_LENGTH/2
char message_frame[msglen]; // The message size would be different, so need a container
char repaired[msglen];    char encoded[msglen + ECC_LENGTH];

RS::ReedSolomon<msglen, ECC_LENGTH> rs;

WiFiClient espClient;
PubSubClient client(espClient);

SoftwareSerial swSer(14, 12, false, 256);

void setup() {
  if (debug) {
    Serial.begin(115200);
  }

  Serial.begin(4800); //Software Serial to Arduino at 4800

  swSer.begin(BAUD_RATE);

  /* Wifi / MQTT code removed to simplify the code */
}

void reconnect() {

  while (!client.connected()) {

    if (client.connect("ESP8266Client")) {
      
      //Request the topics to subscribe to arduino

      char message[] = "<TOPICS?>";
      memset(message_frame, 0, sizeof(message_frame));        // Clear the array
      for(int i = 0; i <= msglen; i++) {    message_frame[i] = message[i];     } // Fill with the message
     
      rs.Encode(message_frame, encoded); 
      swSer.print(encoded);

    }
  }
}

void callback(char* topic, byte* payload, unsigned int length) { // Not currently used 
  }

void loop() {

  if (!client.connected()) {
    reconnect();
  }

  client.loop();

  if (swSer.available()) {

    static int bufpos = 0;
    char inchar = swSer.read();

    if (inchar != EOPmarker && inchar != SOPmarker) {
      serialbuf[bufpos] = inchar;
      bufpos++;

    } else if (inchar == SOPmarker) {
      bufpos = 0;

    } else {

      serialbuf[bufpos] = 0;
      bufpos = 0;

      char* header = subStr(serialbuf, "!", 1);
      char* content =  subStr(serialbuf, "!", 2);

      // Code to test the header then subscribe - Instead, command sent back for test purpose

      char message[msglen] = "<H: ";

      strcat(message,header);
      strcat(message," / C: ");
      strcat(message,content);
      strcat(message,">");
      
      memset(message_frame, 0, sizeof(message_frame));        // Clear the array
      for(int i = 0; i <= msglen; i++) {    message_frame[i] = message[i];     } // Fill with the message
     
      rs.Encode(message_frame, encoded); 
      swSer.print(encoded);
    }

  }
}

char* subStr (char* input_string, char *separator, int segment_number) {
  char *act, *sub, *ptr;
  static char copy[MAX_STRING_LEN];
  int i;
  strcpy(copy, input_string);
  for (i = 1, act = copy; i <= segment_number; i++, act = NULL) {
    sub = strtok_r(act, separator, &ptr);
    if (sub == NULL) break;
  }
  return sub;
}

(Currently, I also have a bug when I have several topics to send : first one return to me "correctly" (with the 2-3% corruption), second one come to me entirely corrupted and third one never arrives... Then the Huzzah resets itself in loop...)

The code is I suppose far from perfect and I'm also interested in your feedback on best practices: by testing different things and copying examples, it ends up being a bit of a mess...
Thank you for taking the time to read me!

Hello Mysterarts,
Welcome to the forum. For following the forum instructions and posting your code correctly on your first post ++Karma;

There are probably people here who will give you a better answer but here goes:

I have 2 ESP8266s (not Huzzah) and my initial attempts to get serial data were painful. There are 2 serial ports, 1 connected via USB and one that only does tx. I tried software serial and got loads of errors and other problems. Then I read somewhere (no, I can't remember where) that SWS does not work on an ESP8266 because the interrupts used to manage the Wi-Fi connection interfere with the timing of the serial port.

Then I discovered:

Serial.swap();

Which makes GPIO15 -> Tx, GPIO 13 <- Rx, as in, it moves the serial port from the usual pins to GPIO 13 and 15. You lose the USB connection, you gain a perfectly good hardware serial port.

I hope that helps.

ESP8266's hardware serial port ought to be full duplex, though I'll admit I haven't played with it much.

The SoftwareSerial on the Uno, however, definitely is only half-duplex. Most software serial implementations can only send OR receive; attempting to do both at once will result in one or both sides seeing garbage characters. It's not entirely clear to me how you're testing this, but if this could explain the issue, well, there's your problem.

If that's the issue: Use a board with a second hardware serial port (Mega, or third party board based on '328pb or '1284p - or a non-AVR board like zero, mkr, etc), or use the hardware serial port and disconnect the ESP8266 while uploading sketches, or adapt the code on the ESP8266 to wait until one character period after the last received character before sending a response and the code on the Uno to not send a second command if it just sent one and is expecting a response.

Note that I don't see an obvious reason that the code you've written would have the uno trying to send and receive at the same time. As an aside, I would probably kick up the baud rate on the software serial port to 9600 or 19200; those are fine for software serial, and reducing the time it takes for a message to be sent, all else being equal, will reduce the length of time during which a collision would occur.

ESP8266's hardware serial port ought to be full duplex, though I'll admit I haven't played with it much.

It is.

Hi,

Sorry for my silence and thank you for your answers!

I was trying your suggestions when my HUZZAH / ESP8266 started acting erratically :frowning:

I just had the time to test the swap() which seems to be promising, thanks again!
But since the week-end I've fighting with my board to make it works, it's very frustrating...

I've asked for help on the Adafruit customer forum: HUZZAH ESP8266 breakout stopped working, keeps reseting - adafruit industries and will maybe take my chance here (but in a new topic because the issue seems very different).

(NB: I also had the time to realize that I absolutely did not use the FEC library in my code above because I placed the end of packet marker before the guardian bytes AND never call the repair function xD)

So yeah, it's was a beginner's mistake: the power supply through the FTDI cable was not stable enough for the ESP (unlike the Arduino 5V pin) :sweat_smile:

So I was able to continue the tests and change my code which now works correctly!

I used the Serial.Swap() on the ESP and I get rid of the serial software which has very unstable results.
I also change the logic of the code to always give the initiative of the communication to the ESP. Every exchange is a message ESP->Uno then an answer Uno->ESP. So it's not really a bidirectional communication as I imagined it (not synchronous) but it's working!

Thanks you again for your answers :slight_smile: