using both serial and newsoftserial

Hi all,
So I have a feeling I'm asking way too much of my little Arduino chip, but a man can dream, can't he?

I've been doing some testing for a much larger set-up. Right now I have an xbee and two arduino pro minis in series: i.e. xbee Tx to Arduino #1 UART Rx, Arduino #1 UART Tx to Arduino #2 UART Rx, Arduino #2 UART Tx to xbee Rx. On #2 I also have a digital compass attached to pins 2 and 3 and am using the "newsoftserial" library to read the data. Arduino #1 initiates the process by reading all the analog pins and writing the packet to its UART Tx, this is read by Arduino #2, and after receiving the line feed byte, reads its own analog pins, adds its data to the Arduino #1 data, adds the extra bytes to make an xbee API packet and sends it off to the xbee via it's UART Tx pin. As soon as it is done, it takes the last complete packet it has received from the compass, adds the extra xbee API bytes, and sends this as well. Right now I have the serial UART @ 115k baud, newsoftserial @ 19.2k. Both the compass and the analog pins are sampled @ 20Hz.

Here's my problem. Currently about 25% of my packets are missing bytes, with the missing bytes always being the data received from the UART Rx pin from the first Arduino. Of the 7 bytes that are coming in (6 data, 1 line feed), I always get the first two bytes and the line feed (since that triggers everything...I could very well be losing the line feed, too, I just haven't set up a way to test that yet). If it's 3 bytes of data, it's always the first 2 and the 4th. If 4, it's the first 2, the 4th and the 6th. As soon as I turn the compass off, I have no problems, no lost bytes presumably since there is nothing to read anymore. This is making me think the UART read and newsoftserial read are interfering. The data from the compass is consistently good, but maybe once every minute the line feed from the compass is missed and I get 2 readings in one packet.

So the question(s) (finally): is it even possible to expect the chip to be able to read a near continuous stream of data from 2 serial ports and not drop bytes? How large of a buffer do the UART serial ports and newsoftserial ports have? I've tried adding delays here and there with no improvement (although maybe I should switch those to time-based triggers instead now that I think about it). I've tried different combinations of "if (Serial.available() > 0)" vs. "while (Serial.available() > 0)" for both types of serial reads with no improvement, either. This is all so I can avoid adding another 70mA to a battery powered system (in the form of an extra xbee and mini to handle the compass data separately)...so I might just have to suck it up especially since the entire system is actually 13 arduino minis! And I'd really like to not go any lower than 20Hz if I can avoid it (yes, I am asking the impossible...I know)...

Anyway...if anyone has any pointers in my quest I would be eternally grateful!

Thank you!
David

Your software based serial ports don't really have a buffer - that's the great thing about built in UART's. Timing with the interrupt solves a portion of that issue, but only a portion.

I'd say you might want to check out the Arduino Mega.

If it's only for that one node, you'll benefit from having 4 built in UARTs, which will eliminate some of your overhead, and allow you to process, then send at full 20Hz.

Depending where in the world you are located, they are still in the relm of reasonable cost. Mine was about $65

Hi Brutus,
Are you certain about the buffer with "newsoftserial"? I thought that was one of the advantages...that and interrupts. But regardless, 90% of my issues are with the UART ports & lost bytes. Seems that as the arduino is reading the soft-serial port I lose incoming data, and rarely the reverse.

The mega is nice, but not really an option here. Thanks, tho, for your pointers.

David

Interesting.. I had assumed the other way around.

As a side experiment, try disabling the software serial and compile without it's library and see if you consistently receive data.

Just a hunch, but I'm wondering if the interrupt itself causes your interruption.

actually I don't even need to recompile...I just power down the device (compass) attached to the soft-serial port (so there is no data to read) and I get hours of good clean data from the UART ports.

That makes sense then.

My hunch is that when the interrupt routine fires, the UART itself is prone to the interruption, and thus the missing data.

I think you are driving the arduino too hard.
The newsoftserial source shows that an entire character is received in an interrupt routine. While that is happening, no other interrupt can occur - so the UART could lose a character.
In the same way, the software tx is done in a routine that has cli() in it; no other interrupts can occur for the 8 + start and stop bits of the tx.
Can you drop your baud rate for the UART port?

The newsoftserial source shows that an entire character is received in an interrupt routine

Yikes! :o (Sorry for the sidebar. Had to get that off my chest.)

Can you drop your baud rate for the UART port?

Or increase the baud rate on the "soft" port?

  • Brian

OK, then that really does support what I am seeing.
I could speed up the soft-serial baud or slow down the UART, but will it really make a difference? In the end, once I reconnect all these mini's, the last arduino is reading ~80 bytes of data from the UART and ~20 bytes from the soft-serial, then writing 2 separate xbee API packets. In other words, time to fire up another xbee and arduino and have that deal with the compass data separately.

Thank you all for your help! Of course if you have a brilliant suggestion, please don't hesitate to keep sharing :slight_smile:

Cheers,
David

Hi David--

Processing a continuous stream at 115K baud is a dicey proposition. Your loop has to be really tight to correctly process ~10000 bytes per second. That gives you only 100 microseconds to process each byte, and if you intersperse that with calls to analogRead() -- which is comparitively slow -- and NewSoftSerial RX at 19.2K baud, I can well imagine that you would have trouble. In fact, I just calculated that each time a character arrives from NSS at 19.2K, it consumes 520 microseconds. During that time, you just lost 3-4 characters coming in from the UART. It all makes sense. I think shellycat is right. That's just too much for Arduino.

Mikal

Thank you for the reply, Mikal. Very helpful knowing a bit more what is happening behind the scenes quantitatively. Before I completely abandon my quest here, I'll play around a bit with the baud rates and see if I can't maybe find a happy medium.

Thank you all again! This has all been very helpful!

Best,
David