Just had a quick view on the code and saw that the outputbuffer is fixed size and a power of 2. This latter makes it easy to do the modulo calculation for the circular buffer.
wrt to the problem, this is typical a requirement error. What do you want it to do? If properly documented any choice is valid (eh well almost).
I see four strategies:
a - if full -> block;
b - if full -> print as many chars as possible;
c - if full -> print no data;
d - if full -> adapt the buffersize
add a) This means no loss of data but loss of program control. It is the choice of the programmer what is most important. In a datalog application failure of one record may or may not be acceptable.
One could implement a library behavior-flag that can be set in the program. This DontLooseData or BlockIfNeeded flag gives the app programmer the choice to change the behaviour and it has only little effect on the librarysize. For me the default value would be true => block if needed, most of the time.
add b) Unpredictable behaviour. There need to be a test before to see if there is enough space for the string, or a test afterwards to see if the string fitted in the buffer. This latter is relative easy to implement.
add c) Similar to 2, but less unpredictable as there are only two outcomes possible. Test afterwards would be my choice. Instead of
J_ASSERT(false, "Serial write buffer full"); one just sets a boolean flag printSucceeded = true; that can be tested. Such a function makes sense anyway (note it differs from writeable_data() )
add d) Increasing the size of the buffers when needed is a strong strategy. One can double the size, add a fixed amount, or add enough to fit the bill. At least it means that the modulo calculation (eg in newhead() ) become slower as one needs the % instead of & . Furthermore it means the use of dynamic memory see http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1230935955
for some discussion.
1) Implement the test function to see if a print("blabla"); has succeeded. It gives back the missing feedback. Just a boolean, or the # chars printed/missed?
2) Add the BlockIfNeeded flag. (see a) This gives the user the choice how the SerLib will behave on a per application basis.
3) Investigate the dynamic buffersize as this frees the programmer from thinking about the buffersize at all. I'm aware this will introduce new problems as the amount of mem is restricted.
PS, Correct me if I'm wrong but on the receiving site is there not a similar problem when more char's are received than fit into the RXbuffer?