Is there better way to remove '\n' character from Serial Buffer while executing Serial.readBytesUntil() function?

I was demonstrating the working principle of atof() function by sending 1234.56 from the InputBox of Serial Monitor and reconstructing it at the UNO side.

Unfortunately, I was seeing 1234.56 and 0.00 on the OutputBox of Serial Monitor. I did execute one more Serial.read() after the Serial.readBytesUntil() to remove the '\n' charcater from the Serial Buffer; but, it did not help.

This is my Sketch:

char myData[8];

void setup()
{
  Serial.begin(9600);   //to enable serial link between PC, SM, IDE, UNO
}

void loop()
{
  byte n = Serial.available();
  if (n != 0) //there is at least one character in Buffer
  {
    byte m = Serial.readBytesUntil('\n', myData, sizeof myData - 1); //'\n' is not saved; m =
    myData[m] = '\0';         //insert null-chracter at the end of string array
    Serial.read();            //to remove '\n' charcater
    //---- reconstruct 1234,56 from the recived ASCII coded 1234.56 -----
    float y = atof(myData);  //atof()= ASCII to float function
    Serial.println(y, 2);
  }
}

Output:

1234.56
0.00

I also tried the following codes which did not work.

Serial.flush();
//------------------
while(Serial.available() > 0)
{
     (void)Serial.read();
}

Ultimately, I had to include the following two lines (which looked odd) near the end of the loop() function of the sketch to clear the Serial Buffer to get rid of 0.00 on the OutputBox of SM.

Serial.end();
Serial.begin(9600);

Would apprciate to know if there are alternative ways of clearing up the Serial Buffer.

If you need to clear all data from the Serial buffer then reading bytes until no more are available should work

Serial.flush() does not empty the input buffer, rather it waits until all data in the output buffer has been read

Of course, if you have control of both the sending and receiving ends of the Serial interface and the format of the data then you can use a different technique that ensures that extra characters are not present in the input buffer in the first place

2 Likes

looks like readBytesUntil() strips the terminating char from the intput and it doesn't appear in the buffer

output for 45.78
an additional read returns 0xffff

 0x34 0x35 0x2e 0x37 0x38 -  0xffff
char buf [90];
char s [20];

void loop()
{
    if (Serial.available ())  {
        int n = Serial.readBytesUntil ('\n', buf, sizeof(buf)-1);

        for (int i = 0; i < n; i++)  {
            sprintf (s, " 0x%02x", buf [i]);
            Serial.print (s);
        }

        char c = Serial.read ();
        sprintf (s, " -  0x%02x", c);
        Serial.print (s);

        Serial.println ();
    }
}

void setup() {
    Serial.begin(9600);
}

What line structure are you sending ?
If \r\n , then receive the input character by character, buffer the chars you want, and ignore or terminate as requitpred on other unwanted characters.

1 Like

Hi @GolamMostafa,

interesting issue ... Your output is related to the size of the buffer and the length of data you input.

I modified your sketch a little bit:

/*
  Forum: https://forum.arduino.cc/t/is-there-better-way-to-remove-n-character-from-serial-buffer-while-executing-serial-readbytesuntil-function/1197775 
  Wokwi: https://wokwi.com/projects/383447769467596801
*/

//char myData[9];
char myData[8];

void setup()
{
  Serial.begin(9600);   //to enable serial link between PC, SM, IDE, UNO
  Serial.println("Input 1234.56 ...");
}

int count;

void loop()
{
  byte n = Serial.available();
  if (n != 0) //there is at least one character in Buffer
  {
    byte m = Serial.readBytesUntil('\n', myData, sizeof myData - 1); //'\n' is not saved; m =
    myData[m] = '\0';         //insert null-chracter at the end of string array
    Serial.read();            //to remove '\n' charcater
    //---- reconstruct 1234,56 from the recived ASCII coded 1234.56 -----
    float y = atof(myData);  //atof()= ASCII to float function
    count++;
    Serial.print(count);
    Serial.print("\t");
    Serial.println(y, 2);
  }
}

The reason for the 0.00 output is that ReadBytesUntil stops reading after "1234.56" [Edit: not plus one byte but due to "sizeof myData -1 = 7" !). So in a next loop() the remaining char in the buffer is read and creates the "0.00" as output from atof().

On Wokwi: https://wokwi.com/projects/383447769467596801

Increasing the buffer size (myData) removes the issue.

Hi @UKHeliBob,

in HardwareSerial it says

void HardwareSerial::end()
{
  // wait for transmission of outgoing data
  flush();

  cbi(*_ucsrb, RXEN0);
  cbi(*_ucsrb, TXEN0);
  cbi(*_ucsrb, RXCIE0);
  cbi(*_ucsrb, UDRIE0);
  
  // clear any received data
  _rx_buffer_head = _rx_buffer_tail;
}

So calling Serial.end() and then Serial.begin() should does empty both buffers.

Are you aware of any known issues with this method?

Generally it would be better of course if HardwareSerial would provide a function that clears the rx buffer in a safe way without this workaround ,,,

readBytesUntil reads the 'until' character

EDIT: unless the input is longer than the buffer

1 Like

Yes! When I enetered smaller number, I experineced the same result as you and there was no 0.00 on the SM after the print of my data item.

"Newline" ('\n') option in the Line ending box of Serial Monitor.

Exactly!

There are seven symbols in my float number (1234.56). So, I declared a string array with dimension eight; where the last bin will hold manually entered "null charcater".

I understand that readBytesUntil() method reads '\n' from Buffer, but it is not saved in the array. (I think the '\n' is put back in the Buffer which causes the UNO to execute the codes again and produces 0.00 output as the method is immediately terminated.)

Yes! If I increase the array size to 9 from 8, the issue goes away. Then, what is wrong with array size 8?

I use readBytesUntil() and it doesn't trigger another read from the Uno.

Here is my code.

void get_input() {
  memset(buf, 0, 16);
  while (!Serial.available());
  
  bufidx = Serial.readBytesUntil(10, buf, 16);
    
  for (byte mod = 0; mod < bufidx; mod++) {
    buf[mod] = toupper(buf[mod]);
    if (buf[mod] == 13 or buf[mod] == 10) {buf[mod] = 0;}
  }
  if ((hlp_B & 0b00000001) == 0) {Serial.println(buf);}
}

I didn't say that didn't. I am not aware of any problems caused by using end() and begin() but it seems clumsy

2 Likes

since you can't control the # of chars entered, shouldn't you creates a buffer that is more than long enough ??

edit: if memory usage is a concern, the buffer can be dynamically allocated within the block where it is used

    if (Serial.available ())  {
        char buf [90];
        int n = Serial.readBytesUntil ('\n', buf, sizeof(buf)-1);

The reason is that your code does not read only until "\n" but also until sizeof myData -1.

So if ( with myData[8] ) you input more than 7 characters the first 7 characters are returned by the function before \n comes. In a second loop() the rest of your characters are taken from the queue.

You can easily check this by transmitting e.g. "123456789" ... :wink:

1 Like

Sorry for any misunderstanding, that was not my intention!

I agree that .end() .. .. begin() is a clumsy way but the developers of HardwareSerial did not provide a proper way to clear the internal buffer (if not done by reading until empty) ...

I don’t work with Strings, but I’m sure you can use your code almost exactly the way it stands, but ‘trim’ the tail after it’s been read, and before processing.

I’m sure there are a couple of tips from String users..

I wouldn't use readBytesUntil at all.

Read the incoming bytes based on
Serial Input Basics - updated - Using Arduino / Introductory Tutorials - Arduino Forum

if you want to sanitize the incoming data, you could check for isPrintable

isPrintable() - Arduino Reference

how would you know when a complete value is entered?

That's easy. Just read char by char until you receive a control character ...

This can be done in loop() without the usual timeout of ReadBytesUntil

https://www.arduino.cc/reference/en/language/functions/communication/serial/readbytesuntil/

I would guess you have done this already, didn't you?

it stops reading when the buffer is full so it doesn't read the rest and the 'until' character \n