timing issue: SoftwareSerial / Hardware Serial / Spektrum satellite receiver

hi there,

through the last days i have been working to read the serial data from a Spektrum Satellite receiver into an ArduinoProMini, 16MHz, 5V, ATmega328, Arduino Software 1.0.
the data comes at 115200 baud, a defined package of 16bytes every 22ms. i have connected the receiver's serial line to the hardware serial RX pin on the arduino board.

to monitor the data, i have established a SoftwareSerial connection (Tx on Pin 6) to the FTDI-interface - serial monitor (have to plug/unplug FTDI everytime after uploading new sketch, but it works).

so now everything is fine - i get accurate data on my screen - IF i set the SoftwareSerial baud rate to 115200 (same as Hardware rate).
BUT if i turn the baud rate of the SoftwareSerial down (which i will later need to do, to be able to send the processed values to a Servo-Interface-board with 38400 baud max), the data read gets inaccurate; the pattern is still recognisable, but bytes are missing or damaged. (see screenshots)

any explanation for this? any workaround? i'd be too happy.
thanks already very much for your time!

best from berlin - jan

ps. here's the simple sketch, that isn't working properly, or does work fine if i tune up the MonitorSerial.begin to (115200)...
also attached the screenshots with the valid and the corrupt data...
xxx

#include <SoftwareSerial.h>
SoftwareSerial MonitorSerial(5, 6); // serial monitor on Pin 6 (only RX)

// serial data from Satellite receiver is connected to RX1 on the
// ArduinoProMini, 5V, 16MHz, ATmega328

int index = 1;
int val[17];
int burp = 0;

void setup()
{
MonitorSerial.begin(38400); // to monitor on screen, or later forward
MonitorSerial.print("Test"); // processed values to Servo-Interface
MonitorSerial.println(); // with max. baudrate 38400

Serial.begin(115200); // serial data rate of the Spektrum Satellite
}

void loop()
{
if (Serial.available() > 0)
{
val[index]= Serial.read();
if (val [index] == 0x01 && val [index-1] >=0)
{
for (index =0; index <17; index++)
{
MonitorSerial.print(val[index]);
MonitorSerial.print("\t");
}
MonitorSerial.print(burp);
MonitorSerial.println();
index=0;
burp++;
}
index++;
}
}

      MonitorSerial.print(burp);
      MonitorSerial.println();

Why?

      MonitorSerial.println(burp);

Outputting data at a slower rate takes longer. I suspect that you are missing incoming data when dillydallying when sending out the data read. When you don't get back to reading the data, the buffer overflows, and incoming data is discarded until there is room in the buffer.

I don't see a reasonable solution to your problem. You simply have more data trying to flow through than the MIDI device can handle.

hi paul,

thanks for coming back. I am not really familiar with serial communication. but from what i understand, the spektrum receiver only sends 16 bytes every 22 milliseconds. that shouldn't be too much input for the buffer to read, keep, put out and do some dillydallying in between?

or does it have to do with my actual 'reading'? is there a way to check that i have read the complete buffer / frame info, before i go to sending the data to the (slower) monitor?

best, jan

The receiver only sends 16 bytes every 22 milliseconds, but the Arduino needs to read those 16 bytes, do some processing (?) AND send those 16 bytes out (a slower connection) in that 22 millisecond time-frame.

hmmm. i am talking 22 milli-, not microseconds... in my calculation, receiving and sending 20 bytes even at low (38400) speed takes each about 0.5ms? (so far, no processing in between)

I believe PaulS called it, more or less, except the problem isn't buffer overflow, but interrupt latency.

The trouble is SoftwareSerial needs to disable interrupts during each byte it transmits. At 38400 baud, each byte takes 260 us (including start+stop bits). The hardware serial port has data coming in at 115200 baud, or 87 us per byte. The actual on-chip hardware UART can buffer 2 bytes. That means interrupts must not be disabled or have response latency longer than 174 us.

Obviously, if SoftwareSerial is disabling interrupts for 260 us every time it sends a byte, you're going to lose incoming data on the hardware serial port. It's not a matter of the buffer in RAM overflowing. The incoming bytes never make it to that buffer, because the hardware serial interrupt was made to wait 260 us, but the UART can't keep up if the interrupt is delayed by 174 us.

This is EXACTLY the sort of problem my new (and not yet well tested) AltSoftSerial library tries to solve. It transmits data using a timer's output compare hardware, so interrupts are only disabled for about 3 us each time the output changes. However, it can only tolerate 1 bit time interrupt latency, or 26 us at 38400 baud. Arduino's interrupt code (in HardwareSerial and the timer0 interrupt for millis) is not very efficient, so it might take longer than 26 us, might not. I do not know.

So you might give my AltSoftSerial a try? I can't say for sure if it will work well at 38400. You can tell me! I see you also send this same message to my email, so I'll reply with the best test code.

Of course, the best solution would be to use a different board which has 2 hardware serial ports (eg, Sanguino), or a board which has a USB port that's separate from the serial port (eg, Teensy).

I took another look at the code generated for Arduino's HardwareSerial interrupts. It's very bad. This bug explains it well, and provides a solution:

http://code.google.com/p/arduino/issues/detail?id=776

If you apply that fix, which is only changing 2 variables to unsigned, then I believe the hardware serial interrupt takes approximately 6 to 7 us. That should be plenty fast enough to allow AltSoftSerial to work at 38400.

hi,

looks like there is indeed a solution: instead of 'SoftwareSerial', i used Paul Stoffregen's brand new beta library 'AltSoftSerial' --> check it out!

don't know exactly what it does (i'm afraid you'll have to refer to Paul for details :wink: ), but the issue with the slowing down of the HardwareSerial input while using a 38400baud serial output were resolved! (thanks again, Paul!)

now going back to further testing... jan