Optimizing UART Communication between Arduino and ESP8266

Hello,

I am trying to establish a simple UART Communication protocol between my Arduino UNO and NodeMCU ESP8266. Right now I have two LEDS connected to two digital pins of the Uno, and the two boards are connected with jumper wires to set up the TX and RX sides. The goal is to be able to make requests to a web server on the ESP 8266 so that it can send a message to the Uno to toggle one of the LEDs, and then receive a response back indicating success. However, while the LED lighting itself is fairly consistent, there is a lot more inconsistency in the Serial messages.

Uno:

#include <SoftwareSerial.h>

#define LEDR 8
#define LEDB 7

SoftwareSerial s(2, 3);

const byte numChars = 32;
char receivedChars[numChars];
bool newData = false;

void setup()
{
  pinMode(LEDR, OUTPUT);
  pinMode(LEDB, OUTPUT);

  digitalWrite(LEDR, LOW);
  digitalWrite(LEDB, LOW);

  Serial.begin(9600);
  s.begin(9600);
}

void loop()
{
  readString();
  if (newData) handleLED();
  newData = false;
  delay(100);
}

void handleLED() {
  Serial.println(receivedChars);
  char c[10];
  strcpy(c, receivedChars);
  Serial.println(c);
  int len = strlen(c);
  int pin = (*c == 'r') ? LEDR : LEDB;
  bool LED_state = !digitalRead(pin);
  digitalWrite(pin, LED_state);

  char c2[20];
  memcpy(c2, c + 2, len - 2); // put id first
  char res[13];
  if (LED_state)
    strcpy(res, "LED is on\n");
  else
    strcpy(res, "LED is off\n");
  Serial.println(res);
  sprintf(c2, "%s %s", c2, res); // append space and state of LED
  s.write(c2);
}

void readString()
{
  static byte ndx = 0;
  char endMarker = '\n';
  char rc;

  while (s.available() > 0 && newData == false)
  {
    rc = s.read();

    if (rc != endMarker)
    {
      receivedChars[ndx] = rc;
      ndx++;
      if (ndx >= numChars)
      {
        ndx = numChars - 1;
      }
    }
    else
    {
      receivedChars[ndx] = '\0'; // terminate the string
      ndx = 0;
      newData = true;
    }
  }
}

ESP:

#include <ESP8266WiFi.h>
#include <ESP8266mDNS.h>
#include <ESP8266WebServer.h>
#include <SoftwareSerial.h>

#define WIFI_SSID ****
#define WIFI_PASSWORD ****

ESP8266WebServer server(80);

SoftwareSerial s(D6, D7);

const byte numChars = 32;
char receivedChars[numChars];
bool newData = false;

void setup()
{
  Serial.begin(9600);
  s.begin(9600);

  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
  Serial.print("Connecting to ");
  Serial.println(WIFI_SSID);
  while (WiFi.status() != WL_CONNECTED)
  {
    Serial.print(".");
    delay(300);
  }
  Serial.println();
  Serial.print("Connected with IP: ");
  Serial.println(WiFi.localIP());
  Serial.println();

  server.on("/", HTTP_GET, handleRoot); // Call the 'handleRoot' function when a client requests URI "/"
  server.on("/LEDR", HTTP_POST, []() -> void
            { signalLED("r"); }); // Call the 'handleLED' function when a POST request is made to URI "/LED"
  server.on("/LEDB", HTTP_POST, []() -> void
            { signalLED("b"); });   // Call the 'handleLED' function when a POST request is made to URI "/LED"
  server.onNotFound(handleNotFound); // When a client requests an unknown URI (i.e. something other than "/"), call function "handleNotFound"

  server.begin(); // start the server
  Serial.println("HTTP server started");
}

void loop()
{
  server.handleClient(); // Listen for HTTP requests from clients
  // handleLED();
  delay(100);
}

void handleRoot()
{
  // When URI / is requested, send a web page with a button to toggle the LED
  server.send(200, "text/plain", "Welcome!");
}

int id = 0; // unique id to mark request id
// run whenever request from app is made
void signalLED(char *c)
{
  char c2[10];
  strcpy(c2, c);
  sprintf(c2, "%s %d\n", c2, id);
  Serial.println(c2);
  s.write(c2);

  while (!s.available()) delay(10);

  readString();
  if (newData)
  {
    char *endId;
    int id2 = strtol(receivedChars, &endId, 10); // collects first part of string and stores location of end in endId
    char res[12];
    strcpy(res, endId + 1);
    if (id2 == id) server.send(200, "text/plain", res); // Send it back to the browser with an HTTP status 303 (See Other) to redirect
    newData = false;
    id++;
  }
}

// run constantly in loop to check if there is a response
void handleLED()
{
  readString();
  if (newData)
  {
    char *endId;
    int id2 = strtol(receivedChars, &endId, 10); // collects first part of string and stores location of end in endId
    char res[12];
    strcpy(res, endId + 1);
    server.send(200, "text/plain", res); // Send it back to the browser with an HTTP status 303 (See Other) to redirect
    newData = false;
    id++;
  }
}

void handleNotFound()
{
  server.send(404, "text/plain", "404: Not found");
}

void readString()
{
  static byte ndx = 0;
  char endMarker = '\n';
  char rc;

  while (s.available() > 0 && newData == false)
  {
    rc = s.read();

    if (rc != endMarker)
    {
      receivedChars[ndx] = rc;
      ndx++;
      if (ndx >= numChars)
      {
        ndx = numChars - 1;
      }
    }
    else
    {
      receivedChars[ndx] = '\0'; // terminate the string
      ndx = 0;
      newData = true;
    }
  }
}

On the Uno side, I get output in the serial monitor like the following:
r 78
r 78
LED is on

r 79
r 79
LED is off

r 80�r 81
r 81
LED is on
The first two chunks print fine, indicating the LED desired, the toggle id, and the status (I am not sure why each r 78 and r 79 line prints twice, but that is not the main issue right now). However, the third is messed up with that question mark symbol and inconsistencies in the id.

Even wilder examples:
image
Regardless of how fast I was firing requests, messed up output like this kept coming up.

On the ESP side, nothing is getting printed to the monitor even though it should be getting printed once the arduino finishes handling the request and sends the response.

Is there any way I can improve this communication / printing? Should I use an alternative to SoftwareSerial? Is my code written in the most appropriate way for such a task? Am I doing anything concerning performance-wise?

Thanks

SoftwareSerial can not do what you want. You can try AltSoftSerial, but it may fail as well.

before attempting to control the LEDs via WiFi did you do a serial basic communications test between the UNO and ESP8266 and get it working?
upload a schematic showing your wiring - in particular the UNO is a 5V logic device and the ESP8266 is 3.3V logic so you require a potential divider on the UNO Tx to ESP8266 Rx signal
any particular reason to use a UNO and ESP8266 - cannot the ESP8266 do all you require?

In your description, you make no reference to having connected the GND pin of the two devices, only TX and RX. Serial depends totally upon having a common ground between the two units, so if you haven't connected it, please do so now.

If that isn't the solution, please produce an accurate depiction of your complete system, either a schematic(greatly preferred), or a (ughly) fritzing diagram, along with a photo of the assembly. This will aid greatly in diagnosing your problem.
Thanks

Apologies, I forgot to upload the diagram.


Bear with me, TinkerCad didn't have a NodeMCU model in the parts list, so assume that the 15 pins on the left side run from 45-b to 59-b and the 15 pins on the right side run from 45-i to 59-i. Pin 2 of the arduino connects to pin D7 of the NodeMCU, and pin 3 of the arduino connects to pin D6 of the NodeMCU. On row 41, the two black wires adjacent to the resistor go to the GND of the arduino and the GND of the NodeMCU.

The resistors are there to protect the NodeMCU, as it can only accept 3.3 V compared to the digital output of 5V from arduino pins. In a simpler diagram, it is like this:

Well 1. I do need more available pins for a future goal of mine and 2. I want to use the Vin pin on the arduino to support a 12 V motor

Are there any other alternatives? Or is there a different overall setup I should use based on people's experiences in the past? I have tried to amalgamate everything I could from Google searches to get what I have so far, but I am all ears for trying something new.

Explain, please. It's an input, so how does it 'support' a motor?

This is for the future, don't worry about it for now. I just meant that the 12 V that is fed into the Uno via the barrel jack can be collected via the Vin pin to spin the motor via a diode-transistor setup.

1 Like

This way you can not collect much current from Vin, nothing for driving a motor.

I have already tried before with my setup and it has worked to its full performance. I want to use it going forward, but this serial communication problem is an issue in terms of my project goals. It seems that it should be simple to set up an effective UART communication between these two boards, so why am I having so much trouble?

One of the issues is that swSerial is not such a good solution usually, but it can be done at the speed you are attempting. Mind you, i never use swSerial on an Esp8266, but rather use the alternate pins that can be assigned to the Uart. You see you are not actually using a uart at all, and wifi functio callback may interfere with swSerial s timing. Please just create a handdrawn schematic rather than tinkercad. The gnd connection between the mcu s is vital and the esp gpio are not 5v tolerant.

Take ESP32. They have 2 serial connections. More memory and are faster.

For the communication between both, you can use JSON. I use it for communication between ESP32 master and 9 ATmega644 (xtal 14,7456MHz better known as communication crystal) as slave with RS485 communication.

You're having so much trouble, I'd suggest trying a simple DC test. Wire an output on one to an input on the other, and verify that you can reliably change a bit at one side and reflect that bit on the LED on the other(use a voltage divider if necessary for the signal path you choose). You've got a very basic system, and it shouldn't be this hard.

Seriously ? Buy a bigger and faster board because something simple isnt working as expected ? Maybe im to minmalistic...

No, it's more because your expectations don't match reality.

The purpose of this comment was not to talk down to you. When you strip a system to it's minimum, often you come across a broken wire, loose terminal, poor solder joint. If not, then the simple test should work, and if not you can investigate further quite quickly because there's so little that could go wrong. I'd say the same for the software side, but frankly, sometimes making blink work across two nodes can be frustrating enough with a wire between them, let alone a comms protocol. Try it. If you can't make a two-wire connection work, then you really need to dig in.

How many times a day is there a topic with software serial that doesn't work?

Why struggeling with a work around as there are possibility, that doesn't cost much and works with guaranty?

Software serial interrupt you're loop. Hardware serial works without interrupting the program.

That i don't know,

why buy another board ? Sure i think that using 2 MCU's can be unpractical, but a solution that just buys more faster boards is not a great option either. In that cause, just get a teensy every time !

That is not always the case, though swSerial is of course way more cpu intensive.

I have reliable enough communication using swSerial on an AVR and hwSerial on an ESP8266, and the ESP8266 does have the alternate pins available if needed. It should work, it can work, and with persistence it most likely will work, with the parts that are already available. Get a more powerful board is not advice that inspires me.

Why it make easy as it can be difficult?