Cannot receive more than 64 bytes sized data via Serial.read()

Projects Programming
1

Hi, when I tested the below code on Arduino Nano 33 BLE,
There is a problem with receiving data only up to exactly 64 bytes.
Data exceeding 64 bytes in length cannot be received.

I've searched many similar topics on google, but nothing helped.
I know that the size of the Arduino serial buffer is 64 bytes,
but I still don't know why the code below doesn't correctly receive more than 64 bytes of data.

What am i doing wrong?
Is it possible to read more than 64 bytes through serial without changing Arduino core library?

#define PIN_BUILTIN_LED_R 22
#define PIN_BUILTIN_LED_G 23
#define PIN_BUILTIN_LED_B 24
#define DebugSerial Serial

#include <Arduino.h>

void setup() {
   pinMode(PIN_BUILTIN_LED_R, OUTPUT);
   pinMode(PIN_BUILTIN_LED_G, OUTPUT);
   pinMode(PIN_BUILTIN_LED_B, OUTPUT);
   
   digitalWrite(PIN_BUILTIN_LED_R, HIGH);
   digitalWrite(PIN_BUILTIN_LED_G, HIGH);
   digitalWrite(PIN_BUILTIN_LED_B, LOW); // LED ON
   
   DebugSerial.begin(9600);
   while(!DebugSerial);
   
   digitalWrite(PIN_BUILTIN_LED_B, HIGH);
   DebugSerial.println("----- SETUP COMPLETE");
}

void loop() {
    bool print_rx_buff = false;
    static char _rx_buff[1024] = { 0, };
    static uint16_t _rx_buff_idx = 0;
    while (DebugSerial.available() > 0) {
        const char new_data = (char)DebugSerial.read();
        if (new_data == '\n') {
            _rx_buff[_rx_buff_idx] = '\0'; // add null terminator
            _rx_buff_idx = 0;
            print_rx_buff = true;
        } else {
            _rx_buff[_rx_buff_idx++] = new_data;
            if (_rx_buff_idx >= sizeof(_rx_buff)) {
                DebugSerial.println("Error: buffer overflow!!");
                digitalWrite(PIN_BUILTIN_LED_R, LOW); // LED ON
                while (1); // halt program
            }
        }
    }
   
    if (print_rx_buff) {
        DebugSerial.print("RX: ");
        DebugSerial.println(_rx_buff);
    }
}

rec

Thanks

2

Your receiver code looks fine. Could you show the transmitter code?

3

The Serial library uses a internal buffer of 64 bytes for most Arduino boards. It is best to read a byte as soon as it arrives, and then the 64 byte buffer gives extra time to do other things in the loop().

You already do that.
The code is working on a Arduino Mega board and a ESP32 in Wokwi simulation with a longer input than 64 bytes. What the problem ?

I'm not used to this kind of programming below. Why the 'const' ?

const char new_data = (char)DebugSerial.read();

At 9600, the bytes come in every millisecond, that is slow. Your processor is fast. In the loop() it sometimes sees a new bytes, but most of the times there is no new bytes yet. You might just as well do a "if" instead of a "while".

When I was writing this, b707 also wrote that the code looks fine

4

the same question from me ....

5

Instead of writing code for the transmitter side, I used Arduino IDE's default serial monitor.
(Check the attached video)

6

I've attached a video of the phenomenon in which the problem occurs.

If you look at the video,
You can see that 64 bytes of data ("00001111...EEEEFFF\n") are received well,
but 65 bytes of data("00001111...EEEEFFFX\n") are not.

7

It looks to me as a bug in Arduino Monitor - it doesn't wait until receiver is busy...

8

What if you print in chunks < 64 bytes or single bytes?
The Stream.print() method may limit array data to fit into the buffer.
EDIT:
Stream has no print() method.
Serial.println() returns the number of bytes sent. Check and report.

9

I think the issue occurs because of a combination of factors.
You read until you find a terminator, but then you write those bytes, and als this takes some time, in which the RX buffer overflows.
easiest solution would be to makes sure that there are no more bytes 'on the way' while you are reading.

while (DebugSerial.available() > 0) {
        const char new_data = (char)DebugSerial.read();
        if (new_data == '\n') {
            _rx_buff[_rx_buff_idx] = '\0'; // add null terminator
            _rx_buff_idx = 0;
            print_rx_buff = true;
        } else {
            _rx_buff[_rx_buff_idx++] = new_data;
            if (_rx_buff_idx >= sizeof(_rx_buff)) {
                DebugSerial.println("Error: buffer overflow!!");
                digitalWrite(PIN_BUILTIN_LED_R, LOW); // LED ON
                while (1); // halt program
            }
        }
    if ( DebugSerial.available() == 0) delay(2); // if there is still a byte coming this will give time to receive it.
    }
1 Like
10

@ooo0o

There is something strange in your video - if your code failed to receive data - it prints "Rx : 0". But I don't see in the code where this message is printed?
According to the code, if it failed to receive "\n" terminator - it should prints nothing at all.

It looks like your actual code in the board is not the same as you showed in the forum?

11

No. the attached video was tested by uploading the code as it was posted in the post.
(i double-checked it)

"RX : 0" message is output by 45-46 lines of the code:

   ...
    if (print_rx_buff) {
        DebugSerial.print("RX: ");
        DebugSerial.println(_rx_buff);
    }
   ...
12

Are you sure?
If the code failed to receive "\n", the print_rx_buff flag still be false and the buffer shouldn't be printed.

13

I've newly attached a complete video including compilation and upload process.

check it out:
https://streamable.com/e/zvtj8a

14

Why do I need a video? It's not that I don't believe you.
Just think - don't you agree with me that with the above code "Rx: 0" should not be printed?

If you think that this printing is normal - please explain, why it printed when print_rx_buff flag is false?

15

Yes, I also think it's definitely abnormal that "RX: 0" is output.
But since this obviously happens,
I posted a question because I wanted to know the cause.

16

Could you test this code on classic Uno/Nano board?

17

consider using Serial.readBytesUntil()

void loop ()
{
    static char _rx_buff[1024];

    while (DebugSerial.available () > 0) {
        int n = DebugSerial.readBytesUntil ('\n', _rx_buff, sizeof (_rx_buff)-1);
        _rx_buff [n] = '\0'; // add null terminator

        DebugSerial.print ("RX: ");
        DebugSerial.println (_rx_buff);
    }
}
18

I just tried it and the same issue occurs: :sob:

image
image1331×730 36.8 KB

19

what is your input string?

20

no.. currently i only have the Nano 33 BLE board.