Several seconds of lag on serial processing

Projects Interfacing w/ Software on the Computer
1

I am using an arduino leonardo to control stepper motors from Visual Studio C++ software on a Win10 laptop. The laptop is connected to the arduino via serial (19200 baud).

At times, the software sends up to 20 commands per second to the arduino (there is at least 50ms between 2 commands, and the arduino processes a command in ~10ms).
During this "high frequency" phase, after several seconds of high responsiveness to commands, the arduino suddenly lags some 3 or 4 seconds behind, misses commands, etc.\

.

Experiment 1:

1- Run the software, waiting for the lag to appear
2- Send a command from the laptop
3- Immediately unplug the serial cable
Seconds after the cable was disconnected the arduino executes the command.

So I thought the problem was with the arduino taking time to process and not the laptop's software. Serial input processing is done in loop() with the following code :

    void loop() {

         if (Serial.available())
         {
          // tried with and without delay :
          delay(10); 
          
          // Tried the 2 following methods of reading :
          
          // METHOD 1
          //char cmdBuffer[15];
          //int lenCmd = Serial.readBytesUntil('.', cmdBuffer, 15);
          //foo(cmdBuffer, lenCmd);
          

          // METHOD 2
          String stringRead = Serial.readString(); 
          foobar(stringRead);

          
          // Serial.Flush has changed and does not flush anymore
          // But without flushing or with Serial.begin(19200); to reset the buffer
          // the lag still appears.
          }
      }

    // setup for completeness:
    void setup() {
         Serial.begin(19200);
         Serial.setTimeout(x); // tried x = 5, 50, 500
    }

I also tried Serial.setTimeout(x) in setup(), with x = 5, 50, 500.
Again, everything works as intended for some time and then instantly breaks.

.

Experiment 2:

1- Run the software, waiting for the bug to appear
2- Unplug-replug the serial cable (not reconnecting the software to the port nor shutting down the arduino)
3- Open the serial monitor of the arduino IDE
4- Communicating with the arduino works perfectly, there is no lag with the commands !

I guess something has gone wrong with what the software sends... Could the arduino slow down commands processing because a kind of spam protection was enabled ? And only on commands sent by the software ??

Pls help, I'm out of ideas :sweat_smile:

2

Please post a complete, actual, compilable sketch.

Consider that the number of posts the forum will allow you on your first day, is limited. To avoid a lengthy question and answer session, please read the forum guidelines about how to streamline your enquiry.
https://forum.arduino.cc/t/how-to-get-the-best-out-of-this-forum/679966

By the way, I suggest you look up 'Serial.flush()'. I'm pretty sure you are mistaken about what it does.

3

What do you think that Serial.flush does? Hint, it has nothing to do with reading the serial port.

The readString() funcion blocks. It will block while waiting for valid input until a timeout. The default timeout is 1 second, I believe. So if you unplug the cable, the valid end of input is not received and it will wait for the timeout.

You may be filling the receive buffer. That will slow reception and you can lose data.

I would suggest that you ditch the readString method and use a non-blocking method like those used in the serial input basics tutorial. Those methods also avoid using the potentially problematic String class.

1 Like
4

I can not post a minimal reproducible example since the source code is over 1000 lines. But there is nothing relevant to serial besides what is posted above. loop() is in the question and setup() is Serial.begin(19200);
Serial.setTimeout(5);

5

I have tried several values of x in Serial.setTimeout(x) : 5, 50, 500. The problem still appears with 5 and 50, and with 500 it is simply too slow. And the lag also appears with the second input method I tried, Serial.readBytesUntil().
For Serial.Flush my bad but removing it makes no difference ( and using Serial.begin(19200); to effectively flush the serial input buffer does not fix my problem either).

6

The begin() function does not clear the serial receive buffer. If you want to clear the serial buffer, use:

while(Serial.available())
{
   Serial.read();  // read and discard all bytes from the receive buffer
}

EDIT: fixed the lack of parentesis after available. @anon57585045 , thanks for the nudge. :slight_smile:

I think that you need hand shaking between the C++ program and the Leonardo so that you don't send too much at a time. Send a command or a few commands an have the Leo signal the C++ program when it is ready for more. Like the grbl firmware works.

7

Surely that is the whole point of an MRE. It concentrates on the code that is causing the problem and in the course of producing it the cause of the problem can often become obvious

1 Like
8

Not 100% sure but isn't it

while(Serial.available() )
{
   Serial.read();  // read and discard all bytes from the receive buffer
}
9

Don't use Strings on an AVR-based Arduino, like the Leonardo. They cause memory problems and unexpected program crashes.

10

But that sketch won't compile because the functions foo() and foobar() are not defined.

Also, I don't know what symptom the sketch is supposed to show. There isn't any output so how do I see the 'lag'?

11

Ok, I finally figured it out. I was using Serial.println() for debugging in the IDE's monitor, and it calls Serial.write() which has this sketchy property:

"As of Arduino IDE 1.0, serial transmission is asynchronous. If there is enough empty space in the transmit buffer, Serial.write() will return before any characters are transmitted over serial. If the transmit buffer is full then Serial.write() will block until there is enough space in the buffer."

The software on the laptop never reads from serial, it just writes commands. I guess this causes the arduino's transmit buffer to saturate and println() to block. My problem was fixed by completely removing serial writes from the arduino's code. (incidentally, I also removed every singe string from the code)

The fact that, once the lag has appeared (ie the transmit buffer is full), from time to time the arduino still executes command is strange though. There is no timeout on Serial.write()'s block and nothing empties the transmit buffer of the arduino, it should not resume operations.
Also, shouldn't the transmit buffer be sent and cleared regularly ? I can't find any info on this.

12

The system UART driver ISR empties it.

Spamming the UART has three possible responses:

  1. block when the buffer is full
  2. stop when the buffer is full, and throw away characters
  3. allow the buffer to expand dynamically with no limit, until it fills memory and causes a system crash

Which one would you choose?

13

I think you misunderstand the purpose of the transmit buffer. Serial.write() does return as soon as all tobe-transmitted data is committed to the buffer. However, that data will then be transmitted irregardless of whether anyone is listening; hence, it does not matter if the PC is reading it's port or not.
However, I note two things - you're using 19200 for the baud rate; that means that at maximum, about 1800 chars can be transmitted in a second, so that's the max throughput for your serial writes. One quick test in these situations is to push that transmit rate up to 115200, or beyond. Your Serial Monitor in the PC won't care, but unless your horridly verbose in your debug print statements, or you put a printout in a tight loop, you won't then encounter delays due to lack of transmit buffer.
The symptom you relate, i.e. intermittent response to commands, tells me your TX buffer is full, write is delaying, and occasionally returning to the code, at which point another read happens, a command executes, and the buffer overflows again. Classic.

14

Yes, and the same choices exist with flow control, because like a full buffer, it demands that the application handle a stream blocking condition.

When a buffer and flow control are combined, the only difference from an application view, is that the interface may block sooner sometimes.

No blocking will ever occur if the output data rate exceeds the input data rate.

15

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.