UART packets loss

Hi everyone,

I am using arduino environment to program the STM32L4 device. To give you brief about my system I have a board with STM32L4 as host controller and ESP connected over TX1,RX1(Serial1 of STM32L4). In my application ESP is sending messages over UART1 but most of my messages gets drop at STM32 reception end. (note - host controller is running with multiple other tasks like doing the sensor data acquisition and publishing the data to ESP over every 512ms.

As per my understanding may be Serail1.available() might not be able to detect the on Receive serial event and messages are getting drooped.(Serial configurations : baud rate :115200, start bit 1, stop bit : 1, parity :none )

Can you please help where can I focus to resolve/understand this UART packets loss issue.

Also - Can anyone let me know in arduino Serial.available() is using any Serial interrupt to check message on UART ? I yes how it is handled, if its not then I would also like to know how this is handled.

Any help or comments are appreciable.

UART sends single bytes, not packages. your ESP is Espressif esp8266? does it run a firmware or your sketch? do you know what buffer overflow and UART 'flow control' is?

Yes I agree UART sends data byte-by -byte (here I mean the same ). I am using the ESP8285. My sketch is running in the ESP. In my case flow control is 0, what do you mean by buffer overflow? do you mean buffer size i am using ? if yes its 4096 bytes.

In general, I send over the serial line, a sentence.

A sentence, for me, can be <!, 1, 62, 530, 466, 530>. The char < and > are sentence begin and end. If I receive serial data without getting a <, I ignore the data in till a < is received and I store the data in till a > is received then I send the data off to be parsed. The ! is used to signify a function to send the data to.

Here is the serial receiver I use on an ESP32; works with the STM32 and Due as well:

/////////////////////////////////////////////////////////////////////////////////////////////////////
void fReceiveSerial_LIDAR( void * parameters  )
{
  bool BeginSentence = false;
  sSerial.reserve ( StringBufferSize300 );
  char OneChar;
  for ( ;; )
  {
    EventBits_t xbit = xEventGroupWaitBits (eg, evtReceiveSerial_LIDAR, pdTRUE, pdTRUE, portMAX_DELAY);
    if ( LIDARSerial.available() >= 1 )
    {
      while ( LIDARSerial.available() )
      {
        OneChar = LIDARSerial.read();
        if ( BeginSentence )
        {
          if ( OneChar == '>')
          {
            if ( xSemaphoreTake( sema_ParseLIDAR_ReceivedSerial, xSemaphoreTicksToWait10 ) == pdTRUE )
            {
              xQueueOverwrite( xQ_LIDAR_Display_INFO, ( void * ) &sSerial );
              xEventGroupSetBits( eg, evtParseLIDAR_ReceivedSerial );
              //
            }
            BeginSentence = false;
            break;
          }
          sSerial.concat ( OneChar );
        }
        else
        {
          if ( OneChar == '<' )
          {
            sSerial = ""; // clear string buffer
            BeginSentence = true; // found begining of sentence
          }
        }
      } //  while ( LIDARSerial.available() )
    } //if ( LIDARSerial.available() >= 1 )
    xSemaphoreGive( sema_ReceiveSerial_LIDAR );
    //            Serial.print( "fReceiveSerial_LIDAR " );
    //        Serial.print(uxTaskGetStackHighWaterMark( NULL ));
    //            Serial.println();
    //        Serial.flush();
  }
  vTaskDelete( NULL );
} // fReceiveSerial_LIDAR( void * parameters  )
//////////////////////////////////////////////////////////////////////////////////////////

Notice the line:

if ( LIDARSerial.available() >= 1 )

I found that using " if ( LIDARSerial.available() >= 0 ) " is not as reliable, with the Due, the STM32 and the ESP32, as using

if ( LIDARSerial.available() >= 1 )

vksgaikwad3:
Also - Can anyone let me know in arduino Serial.available() is using any Serial interrupt to check message on UART ? I yes how it is handled, if its not then I would also like to know how this is handled.

Any help or comments are appreciable.

If using the STM32 and or the ESP32 using freeRTOS and a hardware timer works better as an interrupt driven serial transmitter/receiver.

Idahowalker: Notice the line:

if ( LIDARSerial.available() >= 1 )

I found that using " if ( LIDARSerial.available() >= 0 ) " is not as reliable, with the Due, the STM32 and the ESP32, as using

if ( LIDARSerial.available() >= 1 )

you should use "Serial.available() > 0" it is the same as "Serial.available() >= 1". "Serial.available() >= 0" is always true

Arduino hardware Serial has RX buffer 64 bytes. the buffer is filled with interrupt. if the buffer is full, bytes are dropped (buffer overflow). available() returns count of bytes in the RX buffer.

If the sending device sends data more quickly than the receiving device can clear the Serial Input Buffer then data will be lost.

Removing (say) 64 bytes from the Serial Input Buffer can be done very quickly - much quicker than bytes arrive. But if the receiving device needs to do some calculations with those 64 bytes (or do some totally different work) it may not get that done in time to clear the following 64 bytes before the buffer overflows.

115200 baud is about 11,000 bytes per second so 64 bytes can arrive in about 5 millisecs.

One solution is to ensure that the sending device does not send more than 64 bytes without being told the receiver is ready for more.

If the message length is longer than 64 bytes, but not huge, then another option may be to modify the HardwareSerial code to make the Serial Input Buffer larger. I did that for one current project by simply making a complete copy of the Arduino IDE with a different name. That way I can easily use either the normal version or the modified version.

...R

HI Robin2,

In my case the incoming messages are of having length <= 1024 bytes to 4096 bytes, What could be the impact of increasing the buffer size >64 bytes and what is the maximum RX buffer I can set ?

vksgaikwad3: What could be the impact of increasing the buffer size >64 bytes and what is the maximum RX buffer I can set ?

Assuming your question is about an Arduino with an STM32L4 microprocessor then I don't know. I am only familiar with the 16MHz Arduinos.

...R

vksgaikwad3:
HI Robin2,

In my case the incoming messages are of having length <= 1024 bytes to 4096 bytes, What could be the impact of increasing the buffer size >64 bytes and what is the maximum RX buffer I can set ?

In your case, you must program for your own buffer, as large as necessary and empty the built-in buffer as quickly as it has a character in it and put the character in your own buffer. When you discover the end of your massive message, then the message will all be in your buffer.

Paul