At the heart of serial communications

Hello,

I've well understood the way the incoming serial message are working, but I still have doubts on how things are working when outputting serial data.

What does the program when "Serial.println("stuff to send")" has been called?

  • does the program stops and wait until the whole serial line has been sent?
  • if so, does incoming data gets lost or does the interrupt routine still works to fill in the incoming serial buffer?
  • does the program continues in parallel to the Serial.println("stuff to send") function? If so how do we know that all the data has been output and the com is free again to output more stuff?
  • Is there an output buffer like the 64 byte input buffer we have with the ATMEGA2560?

My major concern is to make sure I don't lose any incoming data while I'm outputting data.
Example: say that I have a device that sends a bunch of stuff on the com 1 at 9600 bauds during 800ms then does nothing the next 200ms. Can I forward this data to another com at 9600 bauds without losing anything?

If anyone has clues on this I'd be happy to hear of it.

Cheers

Baptistou

Hi, welcome to the forum.

This is about a real hardware UART in the microcontroller right ? Let's not go into SoftwareSerial.

There is an output buffer and input buffer in the Serial library. The data is received and transmitted with interrupts.
While transmitting, the code continues, and data can be received without missing something. Everything will run smooth and without glitches.

I think the Serial.flush() function can be used to wait until everything is transmitted. You have to read the reference, it's function was changed.

There is one thing that you must be aware of : When the transmit butter is full, the Serial.prinln() functions will wait until a new byte is available in the buffer. That means that a sketch can suddenly become 10 or 100 times slower, with a full TX buffer.

Things are very different for the other communication hardware. The SPI transmit function transmits it right away and returns if that is finished. The I2C functions Wire.endTransmission and Wire.requestFrom wait until the data is transmitted or received because that is how the library is written, there is no need to wait.

Hi Peter,

Sorry for the late reply, I have been away for a while.
Your explanations are quite useful, but I still have a question:
How can I know how filled is the output buffer? How can I check that it's not full?
I understand the output buffer has the same size as the input one thus 64 bytes for the Atmega. I usually have 100 bytes sentences to transmit. I could split them in two, but when to send the second one?

Again, thanks for your help.

Baptistou

As far as I know it is not possible to check if the buffer is full. Look at the sketch, the baudrate, the bytes transferred and make a guess ::slight_smile:
The delay in between depends on the baudrate. There is a start bit and a stop bit, so rule of thumb is 10 bits for a byte.
With 9600 baudrate, then 960 bytes are transmitter per second.
Suppose 50 bytes are written, then how long do you have to wait for the next 50 bytes ?
I hope this calculation is okay :confused:
TX size is 64. Fill with 50 bytes. There are still 12 bytes available in the buffer, but 50 bytes are needed. So wait until 38 bytes are transmitted.
960 bytes per second is about 1.04 ms per byte. 38 bytes = 40ms.
Conclusion: You would have to wait 40ms.

At 115200 baud, it would be 4ms.
With the Arduino Leonardo, transmitting data to the computer if even faster. The baudrate is kind of ignored and bytes are free running over the usb cable.

The size of the TX buffer is set in HardwareSerial.h
https://github.com/arduino/Arduino/blob/master/hardware/arduino/avr/cores/arduino/HardwareSerial.h
Here is how to change that : Increasing Arduino Serial Buffer Size - Installation & Troubleshooting - Arduino Forum

However, if you update to a new Arduino version, it will be 64 again :cry:

A sketch to test the delay.

// Detect waiting for hardware serial tx buffer
// Tested with Arduino Uno with the normal ATmega16u2 usb-serial chip.

// Trying to set the SERIAL_TX_BUFFER_SIZE here or use #pragma did not work

const char fiftybytes[]   = "01234567890123456789012345678901234567890123456789";
const char hundredbytes[] = "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789";

void setup() {
  Serial.begin(9600);             // 9600 is very slow

  Serial.print("The TX size is: ");
  Serial.println(SERIAL_TX_BUFFER_SIZE);
  Serial.flush();           // wait until everything is transmitted             

  // The TX buffer is now completely empty.
  
  unsigned long m1 = micros();
  Serial.write(hundredbytes, 100);
  unsigned long m2 = micros();
  Serial.println();

  Serial.print("That took: ");
  Serial.print( m2 - m1);
  Serial.println(" us");
}

void loop() {
}

baptistou:
Hi Peter,

Sorry for the late reply, I have been away for a while.
Your explanations are quite useful, but I still have a question:
How can I know how filled is the output buffer? How can I check that it's not full?
I understand the output buffer has the same size as the input one thus 64 bytes for the Atmega. I usually have 100 bytes sentences to transmit. I could split them in two, but when to send the second one?

Again, thanks for your help.

Baptistou

int freeInBuffer = Serial.availableForWrite();

will tell you the space available for write on Serial.

Chuck.

Serial.availableForWrite() 8) Cool. It was added a year ago.

void loop() {
  if (Serial.availableForWrite() >= 50) {
    // transmit next chunk of 50 bytes.
    ...
  }
}

Hello,

That function Serial.availableForWrite() is definitely a must have! I don't know why it is not documented yet. It is the tool I was looking for as my code doesn't want to be slowed down by serial outputs to be flushed.

Now for those who want to calculate how long the data transmission will take on a hardware serial port of an Arduino I think a good approach is:

t = (2400 / rate) * (10000 + 2400 * nb)

where:

t is the time in microseconds
rate is the baudrate
nb is the number of char (bytes) to transmit

This is valid for a com port setting of 8 data bit, parity none, 1 stop bit.

I calculated this function with measurements I made, it gives results slightly higher than the reality (calculated t = more or less 101% of real t).

Thanks again Chucktodd for your useful function!