I want to use this topic to share an observation I made these days.
I made a test program that does nothing but calling a routine that asks serial.available() and calls off all values using serial.read. There is an external source that emits serial data with 9600baud. I recorded with millis() a timestamp and how much data was .available(). 9600Baud equals appx 1Byte/ms.
Despite this routine was called every few ms on average, actual data was available very seldomly! The data came in huge chunks, between 30 and even 120 bytes at once and long stretches with 0 bytes ready. That means that the serial buffer must be much larger than the 64bytes I read somewhere. And the large stretches (>~100ms) without data make it a more difficult to decide if a data block has ended.
Maybe someone can elaborate why the internal gears of the Arduino collect that much data until it is shown as available for retrieving.
That means that the serial buffer must be much larger than the 64bytes I read somewhere
You can read the source.
Maybe your unpublished method is open to misinterpretation.
I made a test program
Please post it here so that we can all try it for ourselves
@alanzalander, why have you hijacked another person's Thread to explore your question?
Please click Report to Moderator and ask to have your Reply #4 and the subsequent Replies moved to your own Topic.
...R
I am still not sure I understand this issue or the answers. If I send one byte, and read that byte when the serialEvent fires. Even if serial is slow and the main loop is running much faster, it should stop firing once the buffer has been emptied by read.
Printing the bytes available shows that the event continues to fire even if there are no bytes left. Which is pretty baffling.
jfrankl8:
I am still not sure I understand this issue or the answers. If I send one byte, and read that byte when the serialEvent fires. Even if serial is slow and the main loop is running much faster, it should stop firing once the buffer has been emptied by read.Printing the bytes available shows that the event continues to fire even if there are no bytes left. Which is pretty baffling.
I think your problem is that Serial.event() is a bit of magic to you, because you do not understand how it works, or misunderstand how it works. It, like so many parts of Arduino as a language, is a little bit of a pitfall. It theory, it's supposed to remove a barrier in code to receiving serial, when in fact, it confuses the beginner, and anyone with a bit more experience doesn't use it. My suggestion is to follow @Robin2's excellent tutorial and eliminate using Serial.event().
jfrankl8:
I am still not sure I understand this issue or the answers. If I send one byte, and read that byte when the serialEvent fires. Even if serial is slow and the main loop is running much faster, it should stop firing once the buffer has been emptied by read.Printing the bytes available shows that the event continues to fire even if there are no bytes left. Which is pretty baffling.
Post your code.
wildbill:
Post your code.
I did. Its in the very first post.
What are you running it on? I just ran it on an Uno and it behaves as expected.
Perehama:
I think your problem is that Serial.event() is a bit of magic to you, because you do not understand how it works, or misunderstand how it works. It, like so many parts of Arduino as a language, is a little bit of a pitfall. It theory, it's supposed to remove a barrier in code to receiving serial, when in fact, it confuses the beginner, and anyone with a bit more experience doesn't use it. My suggestion is to follow @Robin2's excellent tutorial and eliminate using Serial.event().
That's fine but I would still like to understand this better. Does Serial.event() not fire in response to a register flag indicating the serial buffer has data in it? Should I not expect it to stop once the buffer has been emptied with read?
wildbill:
What are you running it on? I just ran it on an Uno and it behaves as expected.
Uno as well. Maybe I am misunderstanding how this is supposed to work. Is my initial thought that it will only fire so long as serial data exists in the queue correct or incorrect?
The code in the (somewhat hidden) main() that invokes serialEvent() on a device with a single Serial port is simply this:
int main(void)
{
init();
initVariant();
setup();
for (;;) {
loop();
if (serialEventRun) serialEventRun();
}
return 0;
}
serialEventRun is simply:
void serialEventRun(void)
{
if (Serial.available()) serialEvent();
}
Regards,
Ray L.
As I understand--
1. When a data byte/character arrives at the RX-section of UART Port, the MCU is interrupter; it goes to an unseen ISR routine, reads the character form RX-section, and saves in the unseen FIFO type serial buffer.
2. In the loop() function, a user may poll the buffer of Step-1 and then reads/prints the character (s) from the buffer.
3. If the SerialEvent() function is declared in the sketch, then it will be automatically called upon by the loop() function provided that a character has arrived in the RX-section. In the SerialEvent() function, the user polls the buffer to read the arrived/stored charter (s).
4. Then what is the advantage of using SerialEvent() function? I see one advantage and that is that the loop() function remains clean from being populated by the codes that poll the serial buffer.
5. Example: To receive/print a "Newline" character terminated string coming form the InputBox of Serial Monitor. (The function automatically puts the null byte in the array that holds the received string.)
char myString[20] = "";
bool serialDone = false;
int i = 0;
void setup()
{
Serial.begin(9600);
}
void loop()
{
if (serialDone == true)
{
Serial.println(myString);
serialDone = false;
i = 0;
}
}
void serialEvent()
{
byte n = Serial.available();
if ( n != 0)
{
char inChar = Serial.read();
if (inChar == '\n')
{
serialDone = true;
}
else
{
myString[i] = inChar;
i++;
}
}
}
Split as requested but alanzalander should read the following for reference !
Could you take a few moments to Learn How To Use The Forum.
It will help you get the best out of the forum in the future.
Other general help and troubleshooting advice can be found here.
Let's start this topic again in its own thread. I was interested in what size chunks the data comes from the serial port. I have here now a program for the purpose to demonstrate this.
int long now;
//--------------------------------------------------
void checkserial() {
static int zeros = 0;
int avail;
byte bytearr[300]; //I have seen over 200 bytes in serial buffer
char s[100]; //for debug output
avail = Serial.available(); //i'll use it multiple times and don't want to see it updating its value here
now = millis(); //for the timestamp
if (avail > 0) { //how many bytes are in the buffer for us? If none, count occurence
Serial.readBytes(bytearr, avail); //read away exactly that many bytes
sprintf(s, "time: %d, zeros before: %d, avail: %d", now, zeros, avail);
Serial.println(s);
zeros = 0; // reset for next bunch of zeroes
} else zeros++; //count zeros only, would be too much text to send a debug output over serial on every occurence
}
//--------------------------------------------------
void setup() {
pinMode(LED_BUILTIN, OUTPUT); //the LED shows you when you can start serial transmission for a test
Serial.begin(9600);
}
//--------------------------------------------------
void loop() {
now = millis();
if ((now > 500) && (now < 3000)) { //LED lights up before going active to give you time to press send
digitalWrite(LED_BUILTIN, HIGH); // When LED turns on start sending serial data with serial monitor or hterm, exchange LOW and HIGH for some boards like ESP8266 Nodemcu !!
} else digitalWrite(LED_BUILTIN, LOW); // turn the LED off, its over, exchange LOW and HIGH for some boards like ESP8266 Nodemcu !!
if ((now > 1000) && (now < 2000)) { //1 second of test with full speed polling
checkserial(); //call function above, check how many bytes are available and make debug output
}
if ((now >= 2000) && (now < 3000)) { //1 second witch a delay in the loop
checkserial(); //call function above, check how many bytes are available and make debug output
delay(130); //let's stress it a little this time and check more seldomly
}
}
If you try it, copy some text, like this source code, into the send line of the serial monitor, After the LED lights up press send.
With my "ESP32 Wifi Kit" I get a response like:
time: 1001, zeros before: 0, avail: 112
time: 1088, zeros before: 22227, avail: 112
time: 1204, zeros before: 29817, avail: 112
time: 1321, zeros before: 29816, avail: 112
time: 1437, zeros before: 29818, avail: 112
time: 1553, zeros before: 29816, avail: 112
time: 1670, zeros before: 29820, avail: 112
time: 1786, zeros before: 29817, avail: 112
time: 1903, zeros before: 29819, avail: 112
time: 2129, zeros before: 24846, avail: 112
time: 2259, zeros before: 0, avail: 224 <---
time: 2390, zeros before: 0, avail: 112
time: 2520, zeros before: 0, avail: 112
time: 2650, zeros before: 0, avail: 112
time: 2780, zeros before: 0, avail: 112
time: 2910, zeros before: 0, avail: 112
In the first second it polls without delay the available bytes and fetches them. Between every poll that deliveres content are ~30000 with 0 available bytes. In the second second (!) I added a delay so that it is barely fast enough to pull all bytes. What one can see is, that there are A LOT more bytes than 64 available regularly. With the delay active even 224 bytes in this example. The hidden serial-ISR code deliveres data apparently in 110ms time frames. With the integrated USB-->serial chip the amount of data is always 112, when I use an external source the response can look like:
time: 1134, zeros before: 34367, avail: 26
time: 1138, zeros before: 957, avail: 4
time: 1255, zeros before: 30098, avail: 112
time: 1350, zeros before: 24270, avail: 90
time: 2259, zeros before: 167510, avail: 100
time: 2389, zeros before: 0, avail: 132
This source sends only a small block of data every second.
With my "ESP8266 NodeMCU" it looks totally different:
time: 1001, zeros before: 0, avail: 117
time: 1001, zeros before: 3, avail: 1
time: 1002, zeros before: 37, avail: 1
time: 1003, zeros before: 38, avail: 1
time: 1039, zeros before: 0, avail: 34
time: 1081, zeros before: 0, avail: 41
time: 1124, zeros before: 0, avail: 41
time: 1167, zeros before: 0, avail: 41
time: 1209, zeros before: 0, avail: 41
time: 1252, zeros before: 0, avail: 41
time: 1295, zeros before: 0, avail: 41
time: 1337, zeros before: 0, avail: 41
time: 1380, zeros before: 0, avail: 41
time: 1423, zeros before: 0, avail: 41
time: 1466, zeros before: 0, avail: 41
time: 1508, zeros before: 0, avail: 41
.....
I don't understand why it somehow synchonizes the loop() runs so that there is no extra call anymore with 0 available bytes. Maybe because of the debug output also going through outbound? But thats another mystery to investigate.
alanzalander:
Let's start this topic again in its own thread. I was interested in what size chunks the data comes from the serial port.
If you have not changed the size of Serial Input Buffer then the answer is any number from 0 to 64 depending on whether anything is sending data and how long it has been since your program last removed characters from the buffer.
Serial.read() removes a single character from the buffer (assuming there is a character in it).
...R
Robin2:
If you have not changed the size of Serial Input Buffer then the answer is any number from 0 to 64 depending on whether anything is sending data and how long it has been since your program last removed characters from the buffer.Serial.read() removes a single character from the buffer (assuming there is a character in it).
...R
I don't know how to change buffer size and I don't want to. But could you please try it yourself. If I see numbers way above 100..200 elements and you say 0..64, it's a bit funny. I think the code above is pretty generic and straightforward. Maybe there is another layer of buffer?
I use .readbytes() in the example but using repeated .read() makes no difference.
The ESP platforms have a 256 byte serial buffer.
See HardwareSerial.cpp and esp32-hal-uart.c
alanzalander:
I don't know how to change buffer size and I don't want to. But could you please try it yourself. If I see numbers way above 100..200 elements and you say 0..64, it's a bit funny.
You are measuring a different thing.
You can receive as many bytes as you want provided that you empty the Serial Input Buffer every time before it overflows. If the data is arriving at (say) 115200 baud you will be getting about 11,500 characters per second so the 64 byte buffer will fill up in about 5 millisecs. If your program empties the data from the buffer every (say) 4 millisecs then you can receive data continuously. On the other hand if something slows your program so it can only empty the buffer every 6 millisecs then you will lose some data.
...R
TheMemberFormerlyKnownAsAWOL:
The ESP platforms have a 256 byte serial buffer.See HardwareSerial.cpp and esp32-hal-uart.c
Then this forms a round picture. Is this true for the ESP8266 also? Is the update interval of ~110ms on ESP32 and ~40ms on ESP8266 also coded there?
There is NO "update interval". Characters are added to the buffer the instant they are received from the external device. They can be read at ANY time after that. But serialEvent will ONLY be called when your code returns from loop(). If your loop() code takes 500mSec to run, then serialEvent can only be called after 500mSec. Any delay you see is in your code, NOT the Serial handler.
delay(130); //let's stress it a little this time and check more seldomly
Regards,
Ray L.