Issues with SoftSerial, AltSoftSerial, I2C, Interrupts – anyone know anything ab

Using Arduino Programming Questions
#1

Hi, I’m new to Arduino and working on a project that will bring in serial data from a sensor at 57600(I have no control over the rate setting), add in additional I2C sensor data, and re-output the combined data out another serial interface. I’ve been testing with a Nano and a Leonardo. The incoming sensor serial data comes in rapid bursts. In terms of total bps it is about 32768bps. I first tried the SoftSerial library to take in data since it has an inverse logic option which is great since I’ll eventually be inserting components that will inverse the logic coming in for the serial sensor data, and rather than have to add even more hardware, it would be great to just invert the logic in software.

So, anyhow, I was able to read the serial sensor data using the SoftwareSerial library, validate it, and re-output out the hardware serial interface without any problems. When I added in the code to read the I2C sensor, every time I read the data, I would lose data (I’m assuming it was on the RX since it wouldn’t make sense losing it on TX out the hardware serial). After doing some reading, I realized that combining SoftwareSerial with I2C is just not an option here because the I2C bus requires interrupts that get in the way.

My understanding of timing, interrupts, and UARTs is completely lacking, and I can’t find a good doc that answers all my questions regarding them – it seems that no one really has a good grasp that’s posted previously. So this is where I’m asking for some help. I’m also wondering what would be the difference behind the scenes if I switch to using the hardware serial port for data input (though I would lost the inverted logic option, and it’s going to make debugging harder). I’m not clear on the steps either way – if I RX using SoftwareSerial, and TX via hardware serial, or vice versa. If RX’ing with SoftwareSerial, what exactly happens during the process of reading on the I2C bus that can cause issues? If RX’ing with hardware serial, does the packet come in, cause an interrupt, get dropped into the pre-defined 64 byte buffer, then just waits until a codepoint gets reached for processing? What would happen if I’m in an I2C bus read at the time? How are interrupts prioritized in this case? What happens if we receive in interrupt while executing an interrupt handler?

#2

On a side note, in the mean time, I thought I would try Paul Stoffregen’s AltSoftSerial library. I got some really bizarre results with it. Basically certain bytes come through fine, and others come through changed or split into two bytes. I completely removed the I2C sensor code out of the sketch, and then completely dumbed the sketch down to just the below, and still have the same results. Note that I cranked up the hardware serial port speed since I’m converting the bytes into numeric decimal format along the way since the additional output would exceed 57600. This might be causing additional issues, but I can see just by swapping out SoftwareSerial with AltSoftSerial with just outputting a y or n as to whether the data is valid that I’m getting bad data. I’m wondering if that’s some sort of weir timing issue with the device that’s sending the data.

#include <AltSoftSerial.h>

AltSoftSerial mySerial;

void setup() {
  mySerial.begin(57600); 
  Serial.begin(250000);
}

void loop() {
byte aByte;
  while (mySerial.available() == 0);
  aByte = mySerial.read();
  Serial.println(aByte,DEC);
}
#3

I realized that combining SoftwareSerial with I2C is just not an option here because the I2C bus requires interrupts that get in the way.

Not true. I2C works totally independently of SoftwareSerial, unless SoftwareSerial is configured to use the SDA and SCL pins (which would be silly).

#4

Does your I2C work fine in isolation (without SoftwareSerial)? Do you have pullup resistors on I2C lines?

#5

OK, thanks florinc. I'm sure I've read some incorrect information and / or interpreted it incorrectly. Doesn't both SoftwareSerial and I2C both rely on interrupts? If they do, doesn't that mean they have some interaction?

The I2C works fine with and without SoftwareSerial. SoftwareSerial works fine without I2C, but when I read I2C it causes an interruption in SoftwareSerial. I'm testing with ADAFruit's accelerometer I2C code. Every time I call readMMA8451, I get lost SoftwareSerial RX data. I've looked through all their code, but it's all very basic definitions and wrapper functions to read and write on the I2C bus.

#6

SoftwareSerial RX is entered at the start of each 10-bit character. It then keeps interrupts disabled while the data is received and exits its ISR approximately halfway through the stop bit. Then the start bit of the next character arrives half a bit width later. At 57600 baud the time between the SoftwareSerial ISR exiting and then starting again is less than 8us. That's not a lot of spare time.

In that brief window, any pending interrupt will be serviced. These include the timer 0 overflow, which keeps milis() counting, the hardware serial TX interrupt and the I2C interrupt. Just servicing an interrupt that does nothing takes a few microseconds. The timer 0 overflow interrupt takes ~5us by itself. It's not hard to imagine that a collision could occur that would cause the RX interrupt entry to be delayed enough to misread the frame.

What I don't understand is why you can't receive and transmit with the hardware UART. Why are you using software serial at all?

#7

Thanks jboyton for clarifying how the softwareserial library works with interrupts. So, doing a little more reading after your pointer there, I see that interrupts of the same type are queued, not ignored. So, it sounds like I do really need to dig into how the wire library reads from the I2C bus. If it uses interrupts, then perhaps it's causing the SoftwareSerial interrupt to get queued instead of processed, and thereby serial data is being lost in that short time span?

The reason I'm not using the hardware serial port is because the end goal is to use a pro micro, which only has one UART. I need 1 TX serial pin and 2 RX serial pins. I'm RX'ing data on one RX port, and TX'ing that same data, along with additional data out the TX. I need the other RX port to receive commands to the Arduino.

It seems like even if I had 2 UARTs, I would still end up with this issue from the I2C bus read.

#8

It seems like even if I had 2 UARTs, I would still end up with this issue from the I2C bus read.

Certainly not. You are not the first to use I2C and UARTS/SoftwareSerial in the same setup. Since setting up SoftwareSerial at a lower baud rate (than 57600 required by your device) is not an option, a hardware serial will definitely help.

#9

I think another UART would solve your problem. It isn't the Wire library, it's the fact that SoftwareSerial has to be executing at a specific time to catch the data. AltSoftSerial is much more forgiving of latency than SoftwareSerial but you may be asking too much for any soft serial. If you need two RX lines at high speed you should rethink your plans for using a micro. Of if you must use a micro then figure out how to add an external UART.

#10

Well I'm wondering now if it's the combination of the I2C wire library using interrupts and using 57600. I dug through the wire libraries and see where the interrupts are used: twi_attachSlaveTxEvent(onRequestService); twi_attachSlaveRxEvent(onReceiveService); and their definitions in twi.c under ISR(TWI_vect)

So, would the benefit of the UART be the capability to dump bytes into a memory buffer even though the system is busy processing an interrupt, whereas you can't do that with the SoftwareSerial library (in the case the system is busy, you would actually lose data with SoftwareSerial if you don't finish with the interrupt handler quick enough), if I'm understanding things correctly?

I was looking at trying a Microduino Core+ for the 2 UARTs in a small form factor, so if that's the case, that seems to be the direction I need to go in.

I wish I could figure out the issue with AltSoftSerial corrupting data coming in. I'm going to do a quick sketch to to dump data from one Arduino at 57600 via hardware serial to another that's receiving at 57600 with AltSoftSerial to see if it's the some peculiarity with the way the device I'm working with is sending serial data, or if it's something with AltSoftSerial.

#11

The UART doesn't place the incoming data in memory. When the data byte is fully received it generates an interrupt and the ISR puts the byte in the memory buffer. The ISR has to be invoked and read the byte before the next one is received.

With SoftwareSerial, the start bit generates an interrupt and the ISR keeps interrupts disabled until the character is almost entirely received. So interrupts are off for over 95% of the time when data is coming in. AltSoftSerial does a much better job. It has to service multiple interrupts per character but it doesn't hold the processor hostage. At 9600 baud AltSoftSerial uses about 5% of the bandwidth. At 57600 it would be higher, maybe too high for your application. It's weird that you couldn't get it to work at all. That might be worth exploring before you buy different hardware, I don't know.

#12

OK thanks for clarifying that process. So the UART isn’t like a UART on a PC, which I was trying to relate it to. From doing some more reading, it seems that typically there is a super tiny hardware buffer on microcontrollers, and in the case of the Arduino, it is two bytes. There’s also nothing giving it easy quick and dirty access to the memory buffer without an interrupt as you explained.

I just created a super simple demo to test AltSoftSerial’s performance. Everything looks good - all the data lines (split by 0A of course) in the serial monitor match up with no discrepancies. So there must just be something about the serial data output from the device I’m testing with that AltSoftSerial doesn’t like unfortunately, and that sounds like a nightmare to troubleshoot, so that means new hardware unless I can get an SPI accelerometer to work along with SoftwareSerial since SPI doesn’t depend on interrupts, well at least it shouldn’t the way I’m implementing it in my application.

Test sketch:
It sends 8 bytes every 2ms (well in addition to whatever time it takes to run through the loops which should be minimal). So, that should be 8 * 8 * 1000 / 2 = 32000 bits, which is well under 57600.

Arduino #1 - Leonardo

int i = 0;
int j;

void setup() {
 Serial1.begin(57600); //Connected to Nano pin 8 clk for AltSoftSerial RX
 Serial.begin(57600); //Just for verification
}

void loop() {
  for (i=0;i<256;i++){
    for (j=i;j<i+8;j++) {
      Serial1.write(j); //Connected to Nano pin 8 clk for AltSoftSerial RX
      Serial.write(j); //Just for verification
      
      i++;    
    }
      delay(2);
  }
}

Arduino #2 - Nano

#include <AltSoftSerial.h>

AltSoftSerial mySerial;

void setup() {
  mySerial.begin(57600); 
  Serial.begin(57600);
  Serial.println("Start");
}

void loop() {
byte aByte;
  while (mySerial.available() == 0);
  aByte = mySerial.read();
  Serial.write(aByte);
}
#13

Why not use a Mega that has 4 hardware USARTs.

A Leonardo has Serial and Serial1 - are you already using both of them so that you also need SoftwareSerial?

From doing some more reading, it seems that typically there is a super tiny hardware buffer on microcontrollers, and in the case of the Arduino, it is two bytes. There's also nothing giving it easy quick and dirty access to the memory buffer without an interrupt as you explained.

This sounds like a description of HardwareSerial - are you having problems with that also ? That seems very unlikely to me.

...R

#14

Just a dumb thought: Is is it possible you're overrunning the RX buffer? What is missing in the data would probably indicate that.

#15

Hi Robin2... I was just using the Leonardo for testing. I need something much smaller for the actual device I'm building, which leaves me with limited options with what's available now with multiple UARTS:(

Jboyton... well what's interesting is that I'm actually getting extra bytes squeezed in, and it's only with certain bytes, not all. With the test sketch above, I can plug the Leonardo TX pin into the Nano's pin 8 for AltSoftSerial and all the bytes pass through fine simulating the same data rate and packet format as the device. When I plug the device into the Nano's pin 8, I get several one or two bytes that are either changed or added per frame. If I switch to SoftwareSerial on the Nano, everything comes through fine (testing without I2C in the mix). I wish I had a scope to see what's coming through or what the difference is so I would know what needed to be tweaked in the AltSoftSerial library.

#16

You could record the incoming data in digital form, or at least a snapshot of it.

Without any soft serial loaded you could setup timer 1 to run at, say, 1us per step. Then start polling the RX line. Whenever RX changes, record the level (0 or 1) in a buffer and read timer 1 and store that in a parallel buffer. A byte has on average 5 or 6 transition. After filling the buffers you could print the data and see the sequence of transitions and the elapsed time between them. This would allow you to reconstruct the incoming data stream. It's a little tricky but it might reveal something.

#17

CircuitSerialKiller: Hi Robin2... I was just using the Leonardo for testing. I need something much smaller for the actual device I'm building, which leaves me with limited options with what's available now with multiple UARTS:(

That does not provide enough information. How many serial connections do you need ?

An Arduino Micro is a small-format Leonardo.

...R

#18

Robin2.. the end goal has been a pro mini for this project. I have a micro, but it's too big for the case it needs to fit in. The only other thing that would fit is a Microduino, but the cost each is 6x that a piece of the pro mini. I wish I could get the AltSoftSerial library working with the device I'm reading. It's going to be hard to find the time to troubleshoot that. So, I'm thinking of doing a few things:

  1. Switch RX for the device over to the UART, then use AltSoftSerial to TX the data out and RX input commands from/to the terminal app, and see how that works. I'm having a problem with the one workstation I'm using detecting the Leonardo bootloader USB device, so I'll have to figure that out. This also means a bit more effort in development to upload sketches if I don't do the dev on the Leonardo since on the Nano I'd be sharing the serial pins between uploading sketches and reading device data.

  2. Switch out the accelerometer to one that I can read on the SPI bus to avoid dealing with interrupts.

jboyton.. thanks for all the technical details regarding serial communications and interrupts with the Arduino. That's all really helpful. I triggered Google's robot detection CAPTCHA about 10 times just trying to look up information on it.

#19

CircuitSerialKiller: I was just using the Leonardo for testing. I need something much smaller for the actual device I'm building, which leaves me with limited options with what's available now with multiple UARTS:(

This could be a job for "The Beetle" -- http://www.dfrobot.com/index.php?route=product/product&path=35_104&product_id=1075#.VavRRazn9dh

#20

CircuitSerialKiller: Robin2.. the end goal has been a pro mini for this project. I have a micro, but it's too big for the case it needs to fit in.

You have still not clearly described what needs to be done and what needs to be connected to what. If you tell us that someone might see a better solution. Focus on WHAT rather than HOW

...R