ESP32 Serial.read to slow

Using Arduino Programming Questions
#1

Hello,

i have an ESP32 and i want to read data from an serial connection, but it seems to be that the Serial2 is to slow because the Serial2.available() always overflows. So I´m losing some bytes while reading.

Is there a way to increase the Serial2 buffer? For now its 256 Bytes big. My receiving message is around 1200 Bytes.

I´m not really understanding what i´m doing wrong because, as mentiond in Robins Post https://forum.arduino.cc/index.php?topic=288234.0 my ESP32 should be faster than the Serial communication.

#include <ETH.h>
#include <WiFi.h>

#define RXD2 36
#define TXD2 4
#define RTS_PIN 2
WiFiServer wifiServer(2001);

static bool eth_connected = false;

void WiFiEvent(WiFiEvent_t event)
{
  switch (event) {
    case SYSTEM_EVENT_ETH_START:
      Serial.println("ETH Started");
      //set eth hostname here
      ETH.setHostname("esp32-ethernet");
      break;
    case SYSTEM_EVENT_ETH_CONNECTED:
      Serial.println("ETH Connected");
      break;
    case SYSTEM_EVENT_ETH_GOT_IP:
      Serial.print("ETH MAC: ");
      Serial.print(ETH.macAddress());
      Serial.print(", IPv4: ");
      Serial.print(ETH.localIP());
      if (ETH.fullDuplex()) {
        Serial.print(", FULL_DUPLEX");
      }
      Serial.print(", ");
      Serial.print(ETH.linkSpeed());
      Serial.println("Mbps");
      eth_connected = true;
      break;
    case SYSTEM_EVENT_ETH_DISCONNECTED:
      Serial.println("ETH Disconnected");
      eth_connected = false;
      break;
    case SYSTEM_EVENT_ETH_STOP:
      Serial.println("ETH Stopped");
      eth_connected = false;
      break;
    default:
      break;
  }
}

void setup()
{
  Serial.begin(115200);
  Serial2.begin(38400, SERIAL_8N1, RXD2, TXD2); // Start Regulator Serial
  Serial2.setTimeout(0);
  WiFi.onEvent(WiFiEvent);
  ETH.begin();

  wifiServer.begin();
}



void loop() {
 
  WiFiClient client = wifiServer.available();
 
  if (client) {
    while (client.connected()) {
      while (client.available()>0) {
        char c = client.read();
        Serial2.write(c);
        //Serial.println(client.available());
        //Serial.println(c, HEX);
      }
      while (Serial2.available()>0) {
        char c = Serial2.read();
        client.write(c);
        Serial.println(Serial2.available());
      }
    }
 
    client.stop();
    Serial.println("Client disconnected");
  }
}

Edit: I´m using a baudrate of 38400, with 19200 everything is working fine.

#2

You are slowing things down by printing multiple bytes for every byte you receive. Serial.println() is asynchronous, but only when there is space in the transmit buffer. Once the transmit buffer is full, Serial.println() blocks until enough data has been transmitted out of the buffer for there to be space for the new data.

#3

Thanks for the fast answer, Serial.println was just for debugging if i remove this line the problem is still there.

void loop() {
 
  WiFiClient client = wifiServer.available();
 
  if (client) {
    while (client.connected()) {
      while (client.available()>0) {
        char c = client.read();
        Serial2.write(c);
        //Serial.println(client.available());
        //Serial.println(c, HEX);
      }
      while (Serial2.available()>0) {
        char c = Serial2.read();
        client.write(c);
        //Serial.println(Serial2.available());
      }
    }
 
    client.stop();
    Serial.println("Client disconnected");
  }
}
#4

I think client.write() is slow. Although the network communication speed is fast, there is overhead to initiate the communication. It is much faster to store the incoming data in a buffer, then write it all at once. If you can't spare the memory for the full 1200 bytes, you could use a smaller buffer and then write whenever the buffer is full.

#5

Thanks, that was the problem, now its also working with 115k. For now everything is working because i´m always expecting 1208 Bytes, but in the future it could be that the response from the serial has a different length. There is also no character which can show me the end of the telegram. So is there a way to speed up the client.write? Or maybe should i just timeout the serial.read ( because now its too fast ) ?

void loop() {
 
  WiFiClient client = wifiServer.available();
 
  if (client) {
    txp = 0;
    memset(SendBuffer, 0, sizeof(SendBuffer));
    while (client.connected()) {
      while (client.available()>0) {
        char c = client.read();
        Serial2.write(c);
        //Serial.println(client.available());
        //Serial.println(c, HEX);
      }
      while (Serial2.available()>0) {
        char c = Serial2.read();
        SendBuffer[txp++] = c;
        //client.write(c);
        //Serial.println(Serial2.available());
      }
      if(txp > 1207) {
        Serial.println(txp);
        client.write(SendBuffer, txp);
        txp = 0;
        memset(SendBuffer, 0, sizeof(SendBuffer));
        client.stop();
      }
    }
 
    client.stop();
    //Serial.println("Client disconnected");
  }
}
#6

I'm not sure I understand your question, but if you add client.write() code inside the while() loop:

     while (Serial2.available()>0) {
        char c = Serial2.read();
        SendBuffer[txp++] = c;
        //client.write(c);
        //Serial.println(Serial2.available());

        if(txp > 1207) {
          Serial.println(txp);
          client.write(SendBuffer, txp);
          txp = 0;
          memset(SendBuffer, 0, sizeof(SendBuffer));
        }
      }
      if(txp > 0) {
        Serial.println(txp);
        client.write(SendBuffer, txp);
        txp = 0;
        memset(SendBuffer, 0, sizeof(SendBuffer));
      }

you can receive any amount of serial input. It will client.write() the data whenever the buffer fills up. This also allows you to use a smaller buffer if you are short on memory (though you do need a large enough buffer that the client.write() overhead doesn't make your while loop too slow to catch all the serial input).

Note that you should only call client.stop() at the end of your while (client.connected()) loop.

I seem to remember that I found in an ESP8266 project I did years ago that there is some maximum number of characters you can send at once, but I don't remember what the exact number was.

#7

Never thought about that way, its not the best way because everything should be stand at once but it works very well. Thanks for your help!

#8

It comes down to whether you know the maximum possible amount of serial input, and whether you have enough memory to hold all that in a buffer. If that's not possible, then you need to send it out in chunks as the buffer fills up. Fortunately, the ESP32 has quite a bit of memory, so that makes the option of sending it all at once more feasible.

I'm glad if I was able to be of assistance. Enjoy!
Per

#9