Serial.read and .readBytes errors and bugs

Projects Programming
1

I want to output 80 chars to an LCD20x4 of whatever comes through Serial.
Easy peacy right?
Not so.
1: using Serial.read() in a while loop most often result in the infamous max of 63 chars, ( using delays can bring it up in the 70 odd chars) .

2: using Serial.readBytes(buf,80); takes up till 80 chars and seemingly treats that buf as a ringbuffer...filling it with the rest! BUT that's not TRUE.
What it does under the hood (I GUESS!!!) is to mark the Serial as not available. So a call Serial.isAvailable() right away returns FALSE even though there are bytes in the buffer.

3: this gives a result that is ODD, because the loop is run multiple times over the buffer reading. Ending with the last reading. So the below program gives this surprising result... but it is not obviously wrong.
the second isAvailable() in readSerial() should/ought to/ be true and put the remaining chars into buf2, but it doesen't.

#include <Wire.h>
#include <LiquidCrystal_I2C.h>
LiquidCrystal_I2C lcd(0x27, 20, 4); // set the LCD address to 0x27 for a 16 chars and 2 line display
static char buf[81] ;
int r = 0;
static char buf2[81];

void clearBuf(){
  memset(buf,0,sizeof(buf));
  memset(buf2,0,sizeof(buf2));
}

// test to verify writing to every position on the screen
void fillScreen() {
  memset(buf,'A',sizeof(buf));
  memset(buf2,'B',sizeof(buf2));
  for (int x = 0; x < 4; x++) {
    for (int i = 0; i < 20; i++) {
      if (i == 0) {
        lcd.setCursor(0, x);
      }
   //   lcd.write((char)(i + 65 + x));
      lcd.write(buf[i+(x*20)]);
    }
  }
}

// trying in vain to up the internal rx-buffer (in the system .h file) : it is not here the limitation is.
void writeSerBufSize() {
  char sb[8];
  itoa(SERIAL_TX_BUFFER_SIZE, sb, 10);
  lcd.setCursor(0, 0);
  for (int i = 0; i < 3; i++) {
    lcd.write(sb[i]);
  }
  lcd.setCursor(0, 0);
}

void setup()
{
  lcd.init();                      // initialize the lcd
  lcd.backlight();
  fillScreen();
  writeSerBufSize();
 
  Serial.begin(9600);
}

int readSerial()
{
  int n = 0;
  if (Serial.available() > 0) {
    n = Serial.readBytes(buf, 80);
    //   delay(10);  // don't you'll lose chars
  } 

// hoping in vain to be able to store the surplus chars in another buffer :-) but it is never called?
  if (Serial.available() > 0 ) {
    r = Serial.readBytes(buf2, 80);
    //   delay(10);   // don't you'll lose chars
  }
 
  return n;
}

//output  stored chars in buf[]

void writeLCD(int n) {
  for (int l = 0; l < 4; l++) {
    for (int x = 0; x < 20; x++) {
      if ((x % 20) == 0) lcd.setCursor(0, l);
      int p = x + (l * 20);
      if (p < n)lcd.write(buf[p]);  // write out buf and not buf2 !!!!
    }
  }
}

void loop()
{
  // when characters arrive over the serial port...
  if (Serial.available()) {
    clearBuf();  
    // anothe try in vain: wait a bit for the entire message to arrive
    //  delay(10);   // don't you'll lose chars 

    // clear the screen
    lcd.clear();
    int n = readSerial();
    writeLCD(n);
    Serial.print("antal char n= ");
    Serial.println(n);
    Serial.println(buf);
    if (r > 0) {   // write whatever surplus chars. Never called. 
      Serial.print("antal char r= ");
      Serial.println(r);
      Serial.println(buf2);
    }
  }
}
2

Double buffer it - have a second 81 byte char array, and stuff the incoming bytes into that and only display it when you've got the full message (either via length, or via start/end marker chars in the serial stream).

3

1: using Serial.read() in a while loop most often result in the infamous max of 63 chars, ( using delays can bring it up in the 70 odd chars) .

I don't know where that "infamous" number comes from but you should explain (and show code) that describes what you mean by this sentence.

2: using Serial.readBytes(buf,80); takes up till 80 chars and seemingly treats that buf as a ringbuffer...filling it with the rest! BUT that's not TRUE.
What it does under the hood (I GUESS!!!) is to mark the Serial as not available. So a call Serial.isAvailable() right away returns FALSE even though there are bytes in the buffer.

Wrong, it reads up to the requested number of bytes from the serial input buffer but returns if the input buffer gets empty. The serial input buffer is a ring buffer, so if you don't empty it often enough you might loose data.

3: this gives a result that is ODD, because the loop is run multiple times over the buffer reading. Ending with the last reading. So the below program gives this surprising result... but it is not obviously wrong.
the second isAvailable() in readSerial() should/ought to/ be true and put the remaining chars into buf2, but it doesen't.

It may be because I'm not a native English speaker but I don't understand what you're trying to tell us. There is no "isAvailable()" method for the Serial object but a "available()" and that returns the number of bytes in the input buffer. This may change between that call and the following read() call because the buffer is filled in the background by an interrupt task that gets the data from the hardware.

4

lcd.setCursor might slow your code down. Can't test at this stage.

If data comes in faster than it can be processed, you have to slow it down.

You can search this site for sterretje xon; I posted a solution using the xon/xoff protocol; you will need a decent terminal program that supports it, e.g. realterm (for windows), but there are many more.

5

estjak:
1: using Serial.read() in a while loop most often result in the infamous max of 63 chars, ( using delays can bring it up in the 70 odd chars) .

Have a look at the examples in Serial Input Basics - simple reliable ways to receive data. You can reliably receive as many characters as you have memory space for. The example assumes 32 characters but you can change numChars to whatever you want.

...R