Non blocking hardware serial

I've got a project where I'm using the Serial object to send output to the serial port. I really can't have that project blocking if the serial port buffer is filled while writing. However, that's exactly how the HardwareSerial class was written. There doesn't seem to be any good way for me to correct this behavior. The Serial object is automatically generated so there isn't a lot I can do to change it other than manually edit the code in the cores folder of the Arduino IDE to fix my problem. That's not good because I can't just edit the Arduino source if I hope to let other people compile the sketch as well. So, what is the correct way to proceed? How do I fix this to allow for non-blocking writes without requiring everyone else to mod their version of Arduino IDE?

What data do you want the system to throw away when you try to send data faster than your data rate can support?

If you don't want your data thrown away you should increase your data rate or reduce your send rate.

Well, here is why: The sketch I have is actually using a hardware timer interrupt as a tick and the main loop checks the tick counter and sends canbus frames out at the tick rate. The serial port is being used for diagnostics right now but the start up text is actually larger than the buffer. SO, what happens is that, on start up, the code blocks at the serial write function until I connect something to the serial port to get the data. Since it blocks, the main loop is not executing and thus the program is doing nothing. I could solve this by not outputting anything over serial until it receives something but it seems like the Arduino library should have had support for allowing characters to be thrown away if you don't care. For instance, it should be possible to run my sketch with the serial disconnected and just let all characters be thrown away. That's normally how serial output works. Making it always block just because nothing is connected is silly. I understand the reasoning (like you said, you don't want it to throw away things if you try to write too fast) but in my case that doesn't apply. It would be nice to be able to connect to the serial at any time and get new serial traffic without having to try to detect when something is listening. This would be super easy to do if I could rewrite the "write" method of HardwareSerial. I'd love to do that but the way Arduino is constructed does not make that easy.

I suppose my point is that blocking should always have been optional.

If your problem is buffer size, why not just make the buffer bigger?

My problem is that a portion of the Arduino core code makes an assumption with basically no way for me to change it. I can't even enlarge the buffer because that's defined in the arduino core as well (so if I change it for my copy it will still have the old value for everyone else who tries to use this project I'm working on.) I think I'm going to have to bypass the transmit buffer and allow for unbuffered sending to the USART through a secondary class. This will cause my code to have to wait while sending strings (70us per character at 115200 baud) but this is more manageable than hanging the code while it waits for a serial connection. That's just not right.

Maybe it is your code or assumptions that's just not right.

I originally didn't think my assumptions were wrong but I think maybe they are. The atmega2560 chip should just keep sending over serial forever no matter if anyone is listening or not shouldn't it? As such, the sketch shouldn't freeze when writing strings to serial, at least not for long. But it seems like maybe it is for me. So, perhaps something is wrong with my code. I'll look at it.

Yeah, I'm pretty sure I was just being dense. The serial comm at the atmega2560 should always keep going no matter what (there really isn't any flow control) so the sketch is not locking up because of that. It's pretty well a false alarm. Sorry to waste time and thank you to people who tried to set me straight.

If I understand your description, you are sending canbus frames from your timer routine
which is call from an ISR.
If you attempt to push data out the serial port and the serial port TX buffer is full, the
serial port write() routine will block waiting for room in the TX buffer.
However, in this situation, it will hang forever because the serial transmit buffer is drained
using interrupts and interrupts are masked because the serial code was being called from
an ISR routine.

--- bill

Hi All,
I have a similar problem. My Arduino program (running on Arduino Due) sends some short information packet every second to a PC through serial port, at 9600 baud, using command "Serial.write(...)" on the main loop. I measured the time required to complete the "write" command: it is nearly the same than the transfer time of the bytes over serial port at 9600 baud (so 1 ms for every sent byte). If I send - for example - a 16 bytes packet, then the main loop blocks for about 16 ms. It is too long for me, as the main loop meanwhile should do some other important tasks on some GPIO ports, requiring fast response time (under 1 ms). So, it seems, that Serial.Write command does not copy the sent bytes to a bigger buffer, that is drained by interrupts later. Instead, it waits for all bytes to be transferred by the USART. (According my timing test, probably there is a software and/or hardware FIFO buffer for 1-2 bytes, but this is too low.) If this is the case, then the real-time nature of the Arduino system is lost if someone uses Serial.Write (at least in the main loop). There is no way to implement a non-blocking, interrupt driven serial port transmit function? I think this can be useful for other users as well.

(running on Arduino Due) ... I measured the time required to complete the "write" command: it is nearly the same than the transfer time of the bytes over serial port at 9600 baud (so 1 ms for every sent byte).

I can confirm from code inspection that the Due "Serial" implementation does NOT use interrupts for transmission.
That's just broken, IMO. The AVR Serial Transmit has been buffered for a long time now (since 1.0?)

I can confirm from code inspection that the Due "Serial" implementation does NOT use interrupts for transmission.

Confirmed.

Maybe make a request over at "Other Software Development"?

I created Due HardwareSerial Transmit not interrupt-driven... · Issue #2271 · arduino/Arduino · GitHub

Thanks for checking it and taking the action!
So it is a DUE-only related issue. (I did not tried Serial.write on other Arduino models.)

Arduino recently added Serial.availableForWrite(), which tells you how many bytes you can transmit without blocking.

I believe it's currently only implemented on AVR, and only in the very latest 1.5.x versions.

1 Like