Arduino stops receiving serial characters into the buffer after some are received (less than 5 in the buffer)

My variable name was different :wink:

I also tried adding a while(!Serial) in setup(), just in case.

could you test this and show the output from the Serial Monitor when you type stuff in?

void setup() {
  Serial.begin(115200); // baudrate does not matter for USB serial
  while (!Serial);
  Serial.println(F("ready"));
}

void loop() {
  static int prevInBuffer = 0;
  static uint32_t heartbeat = millis();
  int avail = Serial.available();

  if (avail != prevInBuffer) { //Print a change in buffer, so we dont spam the monitor
    Serial.print(F("Buffer changed from "));
    Serial.print(prevInBuffer);
    Serial.print(F(" bytes to "));
    Serial.print(avail);
    Serial.println(F(" bytes."));
    prevInBuffer = avail;
  }

  if (avail >= 4) {
    Serial.println(F("Buffer above 4, emptying"));
    for (int i = 0; i < avail; i++) Serial.print(char(Serial.read()));
    Serial.println();
    prevInBuffer = 0;
  }

  if (millis() - heartbeat > 1000) { //did the Arduino crash?
    Serial.print('*');
    heartbeat = millis();
  }
}

Your sketch can actually be reduced to

void setup() {
  Serial.begin(115200); // baudrate does not matter for USB serial
  while (!Serial);
}

void loop() {
  Serial.println(Serial.available());
}

Sending a bunch of bytes results in Serial Monitor printing e.g. 10.
Sending more bytes after that and Serial Monitor still keeps on printing 10.

@sterretje
I'm using IDE 2.0.3, so as you indiciate I never got an issue with the serial monitor.

@J-M-L
Duly noted. There is code for this on the sender's side, and maybe I can add something on the Arduino that throws an error if it gets a start or end marker where it doesn't expect one

I'm not getting any serial monitor lockup with IDE 1.8.19 running on linux.

The OPs bug report on github has received a response, you can read the details there https://github.com/arduino/ArduinoCore-avr/issues/516
The problem comes down to the serial communications being broken up among several USB packets, with Serial.available() only being able to show the number of characters available in the current packet, and no way of knowing about any USB communications pending from the computer. Apparently the Arduino core does not implement a Serial buffer in ram on the Leonardo, so there is no way to tell how many serial character are queued up for sending over USB, and the current USB packet is not cleared until the currently available characters are actually read.

OK then you should be fine. Can the arduino know if a end marker is in the right place or not (eg does the arduino know the size of the expected payload)?

Oh, yeah definitely agree. That's way more concise, but unlike the code I submitted on github it doesnt show whether the buffer isnt filling or Serial.available() isnt changing

what if you add a delay(1); to not overflow the Serial port

void setup() {
  Serial.begin(115200); // baudrate does not matter for USB serial
  while (!Serial);
}

void loop() {
  Serial.println(Serial.available());
  delay(1);
}

does it change anything?

They did excellent detective work! Seems to me less like a bug and more like an undocumented limitation that may have a potential workaround/fix. The fact that no one before me reported this issue makes me genuinely amazed!

I think implementing my own serial buffer using Serial.read (or potentially Serial.peek) might work well

The received payload is 2 bytes, send may be larger when I get around to implementing it. To change the recieve payload youd need to change the uint16_t to uint32_t, which my specific application doesnt need

This is not sending what you think it sends. That only sends 'o' and not a "oo".

Serial Input basics.

This is psuedocode of a python script on a raspberry pi. I did implement it to actually send both characters.

As we established here, this method doesnt work. My latest pseudocode which fully works when sending to leonardo looks like:

all_together = convert_to_a_byte_array('o','(','o','o',')')
send(all_together)

I've never personally noticed it, mainly because I generally don't write code looking for a specific number of characters in the buffer, instead reading characters until a newline is received. The problem is also limited to boards with native USB ports, and may not be present in non-avr board packages (particularly on boards with ample ram where the addition of an Rx buffer is less consequential).

I suspect that most of us implement our own serial buffer and hence haven't observed the problem :wink:

curious if anyone can test

void setup() {
  Serial.begin(115200); // baudrate does not matter for USB serial
  while (!Serial);
}

void loop() {
  Serial.println(Serial.available());
  delay(1);
}

that would still be print 1000 times per second on the USB so quite a flow of data.

I would be also curious to see what this does

void setup() {
  Serial.begin(115200); // baudrate does not matter for USB serial
  while (!Serial);
  Serial.println(F("ready"));
}

void loop() {
  static int prevInBuffer = 0;
  static uint32_t heartbeat = millis();
  int avail = Serial.available();

  if (avail != prevInBuffer) { //Print a change in buffer, so we dont spam the monitor
    Serial.print(F("Buffer changed from "));
    Serial.print(prevInBuffer);
    Serial.print(F(" bytes to "));
    Serial.print(avail);
    Serial.println(F(" bytes."));
    prevInBuffer = avail;
  }

  if (avail >= 4) {
    Serial.println(F("Buffer above 4, emptying"));
    for (int i = 0; i < avail; i++) Serial.print(char(Serial.read()));
    Serial.println();
    prevInBuffer = 0;
  }

  if (millis() - heartbeat > 1000) { //did the Arduino crash?
    Serial.print('*');
    heartbeat = millis();
  }
}

as it is less chatty

any taker ? (I don't have a USB arduino with me to test out)

EDIT: just read the explanation on USB Serial on 32U4 stops incrementing Serial.available() if serial buffer has ≥2 bytes in it and more are received · Issue #516 · arduino/ArduinoCore-avr · GitHub ➜ no need to test then

Do you do so because you are solving a problem, or because you think it should be done that way? If this bug did not exist, what problem would my code have?

The way you worded your reply makes me feel as if I don't understand how to code. If a hardware buffer is present and works correctly, why waste cycles making your own buffer? If you are dealing with a lot of data or long strings, fine. I see it. But all I need is one 16bit binary number 10-100 times a second

The thing is it’s always a challenge to second guess the timing of an asynchronous protocol so we tend to read bytes as they come and remove them from the buffer rather than face a sudden buffer overflow because lots of data came in in one go.

But fair enough - available() should be true to its specification

It is very common for messages to be of varying length and/or longer than can be accommodated in the Serial buffer so it makes sense to read the bytes as they arrive and deal with them until a termination character is read

There are, of course, built in functions to do this in some cases Serial.parseInt()