An alternative Serial Library for Arduino 1.0

I have posted an early version of a new serial library Google Code Archive - Long-term storage for Google Code Project Hosting.. The file is SerialPortBeta20111230.zip.

Please try the SerialPort library and post any comments here.

The SerialPort library provides flexible control of RX and TX buffering.

I used a parametrized template so buffering is specified in the class constructor.

SerialPort<PortNumber, RxBufSize, TxBufSize>

In this example, a SerialPort object is created for port zero, with no buffering so it uses very little RAM.

#include <SerialPort.h>
SerialPort<0, 0, 0> NewSerial;

void setup() {
 NewSerial.begin(9600);
 NewSerial.write("Hello World\r\n");
}
void loop() {}

This example has a RX buffer with a capacity of 32 bytes and unbuffered output:

#include <SerialPort.h>
SerialPort<0, 32, 0> NewSerial;

void setup() {
 NewSerial.begin(9600);
 NewSerial.write("Hello World\r\n");
}
void loop() {}

Flash use varies depending on buffering options. It is usually about the same as Arduino 1.0 HardwareSerial but may be more or less. I will try to optimize flash usage in a later version.

I have attempted to maintain compatibility with the API for Serial.

flush() behaves like HardwareSerial::flush(). On 1.0 and above it waits for TX data to be sent and on Arduino 0023 and before it discards RX data.

I have also added separate flushRx() and flushTx() functions.

feature request based upon "mode com1:9600,N,8,1"
In words, please make it possible to define parity (None,Odd,Even), number of databits (4,5,6,7,8) / stopbits (0,1,2)

Is this possible?

usage:
Sometimes I have to communicate just nibbles (4bit) e.g. a single digit (0..9) or 4 boolean states. And then a byte has 50% overhead.
Some older devices (e.g. terminals) use 7 databits and multiple stopbits (even 1.5 was possible IIRC).

If time permits I'll dive into your code to see if it makes sense to me :wink:

struct SerialRingBuffer {
  uint8_t* buf;   /**< Pointer to start of buffer. */
  uint16_t head;  /**< Index to next empty location. */
  uint16_t tail;  /**< Index to last entry if head != tail. */
  uint16_t size;  /**< Size of the buffer. Capacity is size -1. */
};

Think buffers larger than 255 bytes are very rare ==> uint8_t head, tail, size; // saves a few bytes and might speed up indexing of the buffer access in the isr()'s


 void flushRx() {
    if (RxBufSize) {
      rxbuf_->head = rxbuf_->tail;
    } else {
    // put correct code here/////////////////////////////////////////////////////////////
      usart_->udr;
      usart_->udr;
    }
  }

I think flushRX must allways make head and tail equal, whether there is a buffer or not. ==>

 void flushRx() {
  rxbuf_->head = rxbuf_->tail;
  }

  void end() {
    // wait for transmission of outgoing data
    while (txbuf_->head != txbuf_->tail) {}
    usart_->ucsrb &= ~((1 << B_RXEN) | (1 << B_TXEN)
                     | (1 << B_RXCIE) | (1 << B_UDRIE));
    // clear any received data
    flushRx();  // <<<<<<<< prevent duplicate code?
  }

So far my 2 cents ...

Adding options for parity, character size, and number of stop bits is a good idea.

Parity generation on transmit is easy.

For receive parity errors it would be difficult to flag each character since I buffer 8-bits. I could set a flag indicating a parity error occurred in some character and the user could check and clear it. Is that adequate?

The avr USART is capable of character sizes of 5, 6, 7, 8, and 9 bits. Nine data bits would be difficult since I use an 8-bit data type for ring buffers.

The USART allows one or two stop bits in a asynchronous mode. I didn't know zero stop bits was allowed for asynchronous serial. I thought there needed to be a stop bit to maintain correct framing. The fact that the start bit has space value and the stop bit has mark value allows frame synchronization independent of the character data pattern.

I have had requests for buffers larger than 255 bytes for data logging at very high speeds. Writing an SD can take as long as 200 milliseconds on rare occasions. At high speeds this requires more than 255 bytes. On a Mega over 2000 bytes of buffering is required insure no data loss at 115200 baud.

You can't access the ring buffer when there is no buffer. It is possible that the ring buffer doesn't exist if BUFFERED_RX is zero.

Here is my new code for end() and flushRx()

void end() {
  // wait for transmission of outgoing data
  flushTx();
  usart_->ucsrb &= ~((1 << B_RXEN) | (1 << B_TXEN)
                     | (1 << B_RXCIE) | (1 << B_UDRIE));
  // clear any received data
  flushRx();
}

void flushRx() {
  if (RxBufSize) {
    rxbuf_->flush();
  } else {
    // empty USART fifo
    usart_->flush();
  }
 }

void flushTx() {
  if (TxBufSize) {
    while (!txbuf_->empty()) {}
  }
}

// empty USART fifo
void SerialRegisters::flush() {
  uint8_t b;
  while (ucsra & (1 << B_RXC)) b = udr;
}

Don't spend too much time looking at the current implementation. I am changing it to optimize flash usage. By default templates generate inline code which can use extra flash. I am making changes so read() and write() are not inline.

For receive parity errors it would be difficult to flag each character since I buffer 8-bits. I could set a flag indicating a parity error occurred in some character and the user could check and clear it. Is that adequate?

Any step forward is OK :slight_smile:

The avr USART is capable of character sizes of 5, 6, 7, 8, and 9 bits. Nine data bits would be difficult since I use an 8-bit data type for ring buffers.

Don't lnow of any device using 9 databits (never used it either) so don't care about that one. The 5,6,7 are still interesting.

The # stopbits support sounds good too! Do you have a link to a PDF describing the USART?
Stopbits come from a time where devices needed time to process the incoming data (think mechanical teletypes), and to be sure to give them enough time. Zero stopbits came in when local buffering removed that need.

I am familiar with your (great!) highspeed SD work so now I understand your 16 bit int choice better (add this rationale in the readme file ?)

You can't access the ring buffer when there is no buffer. It is possible that the ring buffer doesn't exist if BUFFERED_RX is zero.

You're right, I missed that in my quick review

New code looks better imho, well done!

Atmel has a number or app notes but I just look in one of the processor datasheets for USART info. The 328 data sheet is here http://www.atmel.com/dyn/resources/prod_documents/doc8161.pdf. Chapter 19 is the USART.

I did some research on stop bits and there must be at least one stop bit. I maintained terminal concentrators in the early 1970s and 1, 1.5, and 2 stop bits were common with 1.5 and 2 used to accommodate mechanical printers like teletypes.

You still need one stop bit for async serial. This is necessary to re-sync the receive clock.

TTL uses logic High (+5 V) for a mark, and logic low (0 V) for a space (hope I remember this correctly).

Since the start bit is space or logic low (0) and the stop bit is mark or logic high (1), there is always a clear demarcation between the previous character and the next one. If there were no stop bit and all data bits were low there would be no way to frame the characters. A sequence of zeros would just hold the line low.

I plan to have a version of begin with a second parameter for options. It will just be bits to set in UCSRC. This will allow even, odd, and no parity. One or two stop bits and 5, 6, 7, or 8 bit characters.

I will save any receive error bits. This will include ring-buffer overrun, USART receive overrun, parity error, and framing error. I will or these bits into a variable that can be read or cleared by new API functions. I think I will only do this for buffered receive in the ISR. I mainly did unbuffered receive for the case where you only want to do output. I need to put the error variable in the ring buffer so the ISR can access it.

I've been playing with this for most of the afternoon and think I've found a problem. I'm accessing the serial port code via Stream* so that I don't know whether it's HardwareSerial or SerialPort. However the Stream overrides in SerialPort aren't marked virtual.

I'm not seeing any buffering here when I write to the serial port.

Checking out the code I see you've gone for inlining everywhere you can. Unfortunately inlining with virtual functions can cause problems, actually taking more code space than when not inlining.

Iain

I didn't mark the overrides as virtual since I didn't want someone to override my functions in a class derived from SerialPort since SerialPort is a template.

I understand the inline problem but this code is a beta prototype to test the API and use of templates for buffer size. It's a pain to move functions out of a template to make them non-inline at this stage. I will optimize flash use later.

I have already totally changed the internal structure by making SerialRingBuffer and SerialRegisters classes and moving most of the code to these classes which don't have inline functions plus it reduces duplicated code for RX and TX. That reduced flash use a lot.

Write is now just these two lines

      // wait for TX ISR if buffer is full
      while (!txbuf_->put(b)) {}
      // enable interrupts
      usart_->ucsrb |= M_UDRIE;

I tested the input buffering buffering in the MegaTest program. It couldn't work if there was no input buffering.

This sketch tests output buffering:

#include <SerialPort.h>
SerialPort<0, 63, 63> NewSerial;

// use macro to substitute for Serial
#define Serial NewSerial

void setup() {
  Serial.begin(9600);
  uint32_t t = micros();
  Serial.println("12345678901234567890");
  t = micros() - t;
  Serial.print("micros: ");
  Serial.println(t);
}
void loop() {}

It writes 20 characters in 220 microseconds so output buffering is working. Here is the output:

12345678901234567890
micros: 220

But not if you access it via Stream*.

Iain

I take that back. I've just written a simple test program to access via Stream* and buffering still happens.

I'd assumed the slowdown in my original app (when I used SerialPort) was due to a lack of buffering. That's not the case so I'll have a good look at my code.

Iain

I will save any receive error bits. This will include ring-buffer overrun, USART receive overrun, parity error, and framing error. I will or these bits into a variable that can be read or cleared by new API functions. I think I will only do this for buffered receive in the ISR. I mainly did unbuffered receive for the case where you only want to do output. I need to put the error variable in the ring buffer so the ISR can access it.

One possible useful function of analyzing the receive error bits is to be able to pass on the rather obscure 'break condition' to a users sketch. The break condition (both sending it and detecting it) is seldom seen or used in the micro-controller world but was used in mainframe and minicomputer systems in the past as a way a remote receiver could 'interrupt' a incoming serial stream by generating the break 'signal' on it's transmit line to the host. When the sending host detected the break condition on it's receive input it would stop transmission and take whatever action the agreed protocol was. I believe the break condition is defined as any space condition lasting longer then a valid frame time length? That is longer then a valid complete character length, but possibly longer?

In the old teletype days a continuous break condition would cause the machine to 'run open' making a very distinctive and noticeable sound that would be noticed by the operators and corrective action taken as it normally meant a broken communications channel.

Lefty

There could very well be bugs so let me know if you find a slowdown problem. I only put two days into this so far. I haven't done much testing.

I put this code out so I could get comments while I am early in development. I really appreciate suggestions like robtillaart's suggestion to add parity, character size, and stop-bit options. Also to return error information for parity, framing, receiver overruns, and buffer overruns.

retrolefty,

I don't see a way to detect break in the avr USART. It probably would give a framing error.

It probably would give a framing error.

Yep, according to the 328 data sheet a 0 stop bit is a framing error.


Rob

fat16lib:
retrolefty,

I don't see a way to detect break in the avr USART. It probably would give a framing error.

Yes, framing error would be the only applicable detection from the existing error codes i believe. So how would one differentiate a purposely sent break condition from a borked character error? Probably can't and maybe one reason one doesn't see the break feature used much anymore. Just wanted to throw out my fuzzy memory of the 'break condition' subject and don't propose that it is all that useful a feature anymore. It was more commonly used in half-duplex comm links as a poor mans 'back channel' method.

Yes, I'm old but my lawn is in good shape so stay off it. :wink:

Lefty

fat16lib:
There could very well be bugs so let me know if you find a slowdown problem. I only put two days into this so far. I haven't done much testing.

I put this code out so I could get comments while I am early in development. I really appreciate suggestions like robtillaart's suggestion to add parity, character size, and stop-bit options. Also to return error information for parity, framing, receiver overruns, and buffer overruns.

OK. It's taken a while to isolate the problem. It seems that creating the second serial port is causing all the problems. This simple program demonstrates the problem. I've no idea what's causing it but if you don't add the SerialPort<1, 16, 16> but it's fine. No idea if the parameters are important. Each loop should report either 0us or 4us. However after the first loop, it reports 156us each time. I'm guessing something in an interrupt is eating all my cycles.

#include <SerialPort.h>

static SerialPort<0, 0, 256> serialPort;
static SerialPort<1, 16, 16> serialPort1;

void setup(void)
{
	SerialPort<0, 0, 256> serialPort;

	serialPort.begin(115200);
	serialPort1.begin(115200);
	for (int route = 0; route < 11; route++)
	{
		uint32_t start, stop;

		start = micros();
		stop = micros();
		serialPort.print(stop - start, DEC);
		serialPort.println("us");
		delay(400);
	}
}

void loop(void)
{
}

Hope that helps.

Iain

Looks like I didn't clean up the code correctly. Remove the SerialPort<> declaration inside setup().

Here's the cleaner version:

#include <SerialPort.h>

static SerialPort<0, 0, 256> serialPort;
static SerialPort<1, 16, 16> serialPort1;

void setup(void)
{
	serialPort.begin(115200);
	serialPort1.begin(115200);
	for (int route = 0; route < 11; route++)
	{
		uint32_t start, stop;

		start = micros();
		stop = micros();
		serialPort.print(stop - start, DEC);
		serialPort.println("us");
		delay(400);
	}
}

void loop(void)
{
}

Iain

I run your example with the version SerialPortBeta20111230.zip on a Mega and get this??

4us
0us
0us
4us
4us
4us
4us
0us
4us
0us
4us

I've run it on two different Mega's and that's not what I get. :-((

Just run it again and it's not going wrong here either. I'll have another look in the morning.

Iain