This is a much larger issue than just ram.
The new HardwareSerial has not only increased ram usage, but increased code usage as well
as changed xmit behavior/timing by forcing all transmits to use an interrupt driven buffer.
IMHO, that is too much to be changing this late in the game.
i.e this has the potential to break existing working code that has worked for years
with lots of subtle hard to find errors from RAM overflows and will cause some projects to no longer fit
inside the AVR, especially on older mega8 and mega168 parts.
Why not provide a backward compatible mechanism that can not only provide
the existing xmit behavior (non buffered xmits) but also provide away to eliminate TX or RX buffers
when not needed as well as provide the new xmit buffering functionality?
One way would be to overload the begin() with a new begin() that allows the user to determine
which directions are enabled and allow the existing begin() to enable the existing
behavior (no xmit buffering). This provides the new functionality without breaking the old behavior
or increasing RAM usage and provides a method to not only eliminate the xmit ram
when not needed but also adds the capability to eliminate the RX ram when it is not needed.
For example, there are some serial devices that only need to receive data. In that case
the RX buffer in the arduino is simply not needed and is wasting AVR resources.
An alternate method to detect the buffer needs would be to change nothing in the API
but then allocate the needed buffer(s) on the first read()/write() using malloc().
Yes there are some other small mods that need to be done in the code to handle things
when there are no buffers allocated. But this would allow a buffer to be allocated only
when it was really needed. So if there were no transmits, no xmit would allocated and
same for RX. It might also allow a larger RX buffer if there were no Xmit buffer needed.
This does not solve the transmit without TX buffer issue (non buffered xmits) and
there might be some lost RX data initially as the RX receiver would not be enabled until
there was an actual read() request.
As far as the code space use by malloc(), nearly half of that can be recovered
by merely changing the head and tail indexes in the ring_buffer structure to be uint8_t rather than ints.
Not only does this save nearly 200 bytes of code, but the code
"as is" really doesn't work anyway with ints anyway
since there are cases where compares and polling is done on them and interrupts are not blocked.
It only works today, because the buffers are smaller than 256 bytes and the upper 8 bits of the int are never used.
In order to really support larger than 256 byte buffers, there needs to be some atomic access to the head/tail
indexes in a few places.
If there is a desire to support larger than 256 bytes buffers, you can still do that with some #if statements
that change the head/tail pointers back to ints and turn on the needed atomic blocking where needed
if the buffer size is larger than 256.
Otherwise, the head/tail indexes might as well be reduced 8 bit values to save the code space - and a few bytes of RAM.
There is also the blocking issue in write()
When the buffer does fill, the write() blocks.
It seems to me that if users don't want' blocking, they may not ever want blocking
and there is no way to avoid this potential blocking.
To me, it seems that if the write is non blocking and returns how many characters that it wrote
that it should not block and return with a 0 if it can't transmit the character.
But that has the potential to create even more subtle bugs as existing code does
not check the return code and users are not accustomed to having to do this or risk
lost xmit data.
These are some of the issues as to why I say that changing the hardwareSerial behavior
to use this new xmit buffering by default is too much too late,
especially given that the full new behavior is forced on everyone with no
way to tweak things to the sketches particular needs or even return to the old behavior and RAM usage
without going in and modifying the HardwareSerial source code.
Worst case a simple define that users could turn on in their HardwareSerial code/header file if they
really need to get back to the non buffered behavior. (to restore timing, reduce RAM and reduce code size)