Is the Serial read or the code slow ?

[GeorgeFlorian]

I am using an ESP32 to read data from a MIFARE Card Reader on Serial1.

// inside void loop()

    if (Serial1.available() > 10)
    {
      // Serial.println();
      // Serial.print("Before: ");
      // Serial.println(Serial1.available()); // number of bytes available to read
      Serial.println("Reading RFID Card...");
      char input[9];

      String str = Serial1.readStringUntil((char)'B');
      // Serial.println(str);

      rf_id = str.substring(3, 11);
      // Serial.println((String)"RFID HEX: '" + rf_id + "' - size: " + rf_id.length());
      rf_id.toCharArray(input, 9);

      // Serial.println(input);
      int val = StrToHex(input);
      // Serial.println(val);

      rf_id = String(val);
      while (rf_id.length() < 10)
      {
        rf_id = "0" + rf_id;
      }
      // Serial.println((String)"RFID DEC: '" + rf_id + "' - size: " + rf_id.length());

      // Serial.print("After: ");
      // Serial.println(Serial1.available()); // number of bytes available to read
      Serial1.flush();
    }

The reader sends:

-49E69599
Block   1 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

All I need is the HEX code 49E69599. I then transform it into a 10 digit DEC number and store into into a String.

Whenever I read a card this is the output:

Reading RFID Card...
RFID: 1239848345 was sent over UDP.
Reading RFID Card...

This means that the Serial1 is still available even though I've used Serial1.flush(); .

1. I would like to speed up the process of reading cards, because right now it's kind of random; sometimes it works pretty fast, and sometimes I have to keep the card in front of the reader for a whole second to get a read.
2. I would also like to discard everything in the Serial1 buffer after the 'B' character.

Basically this: String str = Serial1.readStringUntil((char)'B'); then discard the Serial1 buffer.

Serial1.flush() basically does nothing, because keeping it doesn't discard the buffer as the "flush" part of it implies, nor does removing it improves anything at all.

Thank you.

[UKHeliBob]

Serial1.flush() basically does nothing, because keeping it doesn't discard the buffer as the "flush" part of it implies, nor does removing it improves anything at all.

Serial.flush() does nothing to the input buffer. It waits until the output buffer is empty

See Serial.flush() - Arduino Reference

If you want to empty the input buffer, read from it until there is nothing available

while (Seria1.available())
{
  Serial1.read();
}

[GeorgeFlorian]

UKHeliBob:
Serial.flush() does nothing to the input buffer. It waits until the output buffer is empty

See Serial.flush() - Arduino Reference

If you want to empty the input buffer, read from it until there is nothing available

while (Seria1.available())

{
 Serial1.read();
}

Thank you. Emptying the buffer will make the read even slower, right ?

[anon73444976]

Emptying, at most, 64 characters?
Not very long at all, compared to the time it took for them to get there.

[GeorgeFlorian]

TheMemberFormerlyKnownAsAWOL:
Emptying, at most, 64 characters?
Not very long at all, compared to the time it took for them to get there.

I've set the reader to send less data on Serial and now it sends only:

013 010
045 065 065 048 052 070 048 050 067
013 010
062

which translates to:

-AA04F02C
>
I've also changed the code to this:
**__ <strong>**    if (Serial1.available() > 10)     {       Serial.println("Reading RFID Card...");       char input[9];       String str = Serial1.readStringUntil((char)'>'); // now reading until '>'       rf_id = str.substring(3, 11);       rf_id.toCharArray(input, 9);       int val = StrToHex(input);       rf_id = String(val);       while (rf_id.length() < 10)       {         rf_id = "0" + rf_id;       }       while (Serial1.available())         Serial1.read();     }**</strong> __**
Now I have 14 characters sent over Serial1 and while this code now prints Reading RFID Card... only once, the read time is still kind of random and slow.

[cattledog]

the read time is still kind of random and slow.

How have you determined this?

if (Serial1.available() > 10)

You start reading when 11 characters are available, but the entire message is 14. Perhaps the time for the last three characters to arrive is random, and in some cases "slow". Perhaps the ESP32 is off performing some wifi maintenance operation.

Try

if (Serial1.available() > 13)

Alternatively, and perhaps better, is to use a non blocking reading procedure, read each character when it arrives, and only work with the input when you have received the '>' and the message is complete.

See Serial Input Basics for ideas.

[drmpf]

Check out my Arduino Software Solutions for how to read text from serial connections.
It includes code non-blocking reads terminated by a specific char and for flushing the input without blocking your loop() code.

To convert your string to hex I would suggest using SafeString hexToLong() converter which wraps the low level c-string method and adds error checking. See the bottom of the Software Solutions page

See SafeStringToNum.ino but replace

int numResult = 0;
  if (sfStr.toInt(numResult)) { // ignores leading and trailing whitespace
    Serial.print(F(" numResult = ")); Serial.println(numResult);
  } else {
    Serial.print(F("Not a valid integer '")); Serial.print(numStr); Serial.println("'");
  }

with

long numResult = 0;
  if (sfStr.hexToLong(numResult)) { // ignores leading and trailing whitespace
    Serial.print(F(" numResult = ")); Serial.println(numResult);
  } else {
    Serial.print(F("Not a valid Hex number '")); Serial.print(numStr); Serial.println("'");
  }

[Idahowalker]

Here is my code of a thread to read serial on an ESP32.

void fReceiveSerial_LIDAR( void * parameters  )
{
  bool BeginSentence = false;
  char OneChar;
  char *str;
  str = (char *)ps_calloc(300, sizeof(char) ); // put str buffer into PSRAM
  // log_i("Free PSRAM before String: %d", ESP.getFreePsram());
  for ( ;; )
  {
    EventBits_t xbit = xEventGroupWaitBits (eg, evtReceiveSerial_LIDAR, pdTRUE, pdTRUE, portMAX_DELAY);
    if ( LIDARSerial.available() >= 1 )
    {
      while ( LIDARSerial.available() )
      {
        OneChar = LIDARSerial.read();
        if ( BeginSentence )
        {
          if ( OneChar == '>')
          {
            if ( xSemaphoreTake( sema_ParseLIDAR_ReceivedSerial, xSemaphoreTicksToWait10 ) == pdTRUE )
            {
               xQueueOverwrite( xQ_LIDAR_Display_INFO, ( void * ) &str );
              xEventGroupSetBits( eg, evtParseLIDAR_ReceivedSerial );
              //
            }
            BeginSentence = false;
            break;
          }
          strncat( str, &OneChar, 1 );
        }
        else
        {
          if ( OneChar == '<' )
          {
            strcpy( str, ""); // clear string buffer
            BeginSentence = true; // found beginning of sentence
          }
        }
      } //  while ( LIDARSerial.available() )
    } //if ( LIDARSerial.available() >= 1 )
    xSemaphoreGive( sema_ReceiveSerial_LIDAR );
    //        log_i( "fReceiveSerial_LIDAR " );
    //        log_i(uxTaskGetStackHighWaterMark( NULL ));
  }
  free(str);
  vTaskDelete( NULL );
} //void fParseSerial( void * parameters  )

I create a buffer for the serial data.
The code checks for incoming data looking for the start character. Once the start character is received, the incoming serial data is received and kept until the end character is received. When the end character is received the received data is sent to the parsing task; another thread.

All non-blocking.

In response to the ESP32 off doing the WiFi thing and not receiving serial... The ESP32 has 2 processors. The main processor has 2 cores. WiFi when its loaded runs on core0 and the program runs on core1. The 2 cores run independently of each other.

[GeorgeFlorian]

cattledog:
How have you determined this?

if (Serial1.available() > 10)

You start reading when 11 characters are available, but the entire message is 14. Perhaps the time for the last three characters to arrive is random, and in some cases "slow". Perhaps the ESP32 is off performing some wifi maintenance operation.

Try

if (Serial1.available() > 13)

Alternatively, and perhaps better, is to use a non blocking reading procedure, read each character when it arrives, and only work with the input when you have received the '>' and the message is complete.

See Serial Input Basics for ideas.

How is the following code non-blocking if you're using a while loop ?

// read Serial until until_c char found, returns true when found else false
// non-blocking, until_c is returned as last char in String, updates input String with chars read
bool readStringUntil(String& input, char until_c) {
  while (Serial.available()) {
    char c = Serial.read();
    input += c;
    if (c == until_c) {
      return true;
    }
  }
  return false;
}
void loop() {
  if (readStringUntil(input, '\n')) { // read until find newline
    Serial.print(F(" got a line of input '")); Serial.print(input); Serial.println("'");
    input = ""; // clear after processing for next line
  }
}

Why is my code blocking if I read from serial only after I've got enough byte to read ?
For this case only I've got 14 characters sent and I use thsi code:

    if (Serial1.available() > 13)
    {
      Serial.println("Reading RFID Card...");
      char input[9];
      // String str = Serial1.readStringUntil((char)'\r');
      String str = Serial1.readStringUntil((char)'>');
      while (Serial1.available())
      {
        Serial1.read();
      }
     }

The only blocking part should be the while loop, but you are using it as well, so why is my code blocking and yours not ?

cattledog:
How have you determined this?

if (Serial1.available() > 10)

You start reading when 11 characters are available, but the entire message is 14. Perhaps the time for the last three characters to arrive is random, and in some cases "slow". Perhaps the ESP32 is off performing some wifi maintenance operation.

Try

if (Serial1.available() > 13)

Alternatively, and perhaps better, is to use a non blocking reading procedure, read each character when it arrives, and only work with the input when you have received the '>' and the message is complete.

See Serial Input Basics for ideas.

By read time I mean the time it takes the reader to read the card. Once it reads the card the whole process is almost instant.
I've put timers inside the methods and it takes less then 1ms to clear the if (Serial1.available() > 10), so that's good.
I've contacted the card reader manufacturer and they replied that "This interval depends on negotiation time between reader and card, not on software.". So there's that.

[drmpf]

How is the following code non-blocking if you're using a while loop ?

You can change it to (edit Not recommended see #11 below)
if (Serial.available()) {
if you like, but usually consuming the available bytes in the RX buffer is quick

if (Serial1.available() > 13)

opps my bad, I missed that. Nice idea for a small known input.

Have you solved your original problem?

Did you try the Input Flushing code? The timeout is 1sec so you will want to reduce that.

[GeorgeFlorian]

drmpf:
You can change it to

if (Serial.available()) {

if you like, but usually consuming the available bytes in the RX buffer is quick

if (Serial1.available() > 13)

opps my bad, I missed that. Nice idea for a small known input.

Have you solved your original problem?

Did you try the Input Flushing code? The timeout is 1sec so you will want to reduce that.

Thank you.

I've contacted the card reader manufacturer and they replied that "This interval depends on negotiation time between reader and card, not on software.". So there's that. It's not a problem from the code.

[drmpf]

Actually if (Serial.available()) { is not recommended
The usual problem is RX buffer overflow so when you come to read the Serial you normally want to read as much as you can to clear the buffer.

Even if you have a 'fast' processor, the rest of you loop() code can be slow, due to hidden delays and
third party libraries and more commonly Serial print statements blocking waiting for TX buffer space.

If you want more details see my Arduino Serial I/O for the Real World tutorial for solutions to these issues