I've been playing with this for most of the afternoon and think I've found a problem. I'm accessing the serial port code via Stream* so that I don't know whether it's HardwareSerial or SerialPort. However the Stream overrides in SerialPort aren't marked virtual.
I'm not seeing any buffering here when I write to the serial port.
Checking out the code I see you've gone for inlining everywhere you can. Unfortunately inlining with virtual functions can cause problems, actually taking more code space than when not inlining.
Iain