Does anybody know a good way to implement a buffer between fast I2C readings and a rather slow, but latency-sensitive UART output?
I have a samd21 based I2C main controller that reads multiple sensor values from other samd21-based I2C peripherals. I use the wire library for I2C.
The main controller checks each sensor value and when it is above a threshold it should output the sensor value to a UART. The UART output should be a steady flow with lowest latency as possible.
As I2C is much faster than this UART, the UART becomes a blocking bottleneck. So I thought to implement a circular (FIFO) buffer in between. But unable to do so because the only way I can think of doing this is to put the I2C wire.read() commands into a timer’s ISR function. That unfortunately is not allowed because Wire depends on interrupts and won’t work inside an ISR.
Anybody know a sophisticated solution?
Here’s my non-working(!) code for reference:
# include <Wire.h>
// circular buffer
const int sensorBufferSize = 64;
struct buffer {
uint8_t sensorNumber;
uint16_t sensorValue;
};
buffer sensorBuffer[sensorBufferSize];
volatile uint8_t sensorBufferHead = 0;
volatile uint8_t sensorBufferTail = 0;
void I2CRequestSensorData() {
Wire.requestFrom(1, 3); // requesting 3 bytes from I2C peripheral with address 1
sensorNumber = Wire.read(); // 1st returned byte is sensornumber
sensorValue = Wire.read() | (Wire.read() << 8); // 2nd and 3rd bytes are sensorvalues
if (sensorValue > 900) { // only when above this threshold
uint8_t nextHead = (sensorBufferHead + 1) % sensorBufferSize;
if (nextHead != sensorBufferTail) { // Check if buffer is full
sensorBufferTail[sensorBufferHead].sensorNumber = sensor;
sensorBufferTail[sensorBufferHead].sensorValue = value;
sensorBufferHead = nextHead;
}
}
}
void processBuffer () {
while (sensorBufferTail != sensorBufferHead) {
sensorNumber = sensorBuffer[sensorBufferTail].sensorNumber;
sensorValue = sensorBuffer[sensorBufferTail].sensorValue;
Serial.print(sensorNumber);Serial.print(": ");Serial.println(sensorValue);
}
}
void timerISR() { // request sensor data
I2CRequestSensorData(); // this is not allowed here. But how to do it correctly?
TC5->COUNT16.INTFLAG.bit.MC0 = 1; // Clear the interrupt
}
void setup() {
Serial.begin(9600);
Wire.begin(); // join i2c bus as master
Wire.setClock(400000); // high speed mode
setupTimer(); // set up timer ISR (function not posted here but triggers every 1ms)
}
void loop() {
processBuffer(); // process the I2C buffer as fast as possible to reduce latency as much as possible
}
*edit: added threshold in I2CRequestSensorData() *
What is the point of the buffer? If you don't increase the speed of the Serial port, you'll have to drop values from the i2c port input.
Why not increase the speed of the serial port?
Not sure what you are trying to achieve.
Serial has its own FIFO buffer and the pace of emitting bytes is governed by the baud rate.
The way the Serial objects work in Arduino is that if the FIFO output buffer is full, print becomes blocking. You can test if there is room for what you want to print by using Serial.availableForWrite() if you don't want to get blocked but you'll need to devise a strategy as long term you'll always end up either not sending data if you don't want to block or blocking the code and increasing the period where you do something else if you can't send out fast enough compared to what you acquire.
of course, using a slow baud rate for Serial does not help....
➜ go for 500,000 or 1,000,000 if you want to speed up things
unfortunately increasing the baudrate is not possible. The device connected to it doesn't allow for faster transmissions. But I just noticed that I missed an important part in my code: there is a threshold check. So I edited it above.
The goal is as follows:
The sensors are checked at fast speed. The peripherals (I have multiple, but in the code above I only address 1 for better readability) do this checking. On every I2C request (Wire.requestFrom(1, 3)) the peripheral is allowed to send data of 1 sensor to the main.
The main controllers checks if the sensors are above a certain threshold value (this part I forgot to copy in the code snippet above). If so, they should be sent over the UART.
On average, the amount of time the sensor values will be above threshold isn't that much. That amount of data can be easily send over a 9600 baud line. However, it fluctuates. so for example sometimes multiple sensors are over-threshold at the same time or sometimes, for a short seconds, a lot of sensors are over-threshold. As it is not allowed to drop sensor values (when above threshold) I should find a way to queue them for transmission over the UART. And I want to keep polling the I2C peripherals for data so I don't want to poll once, process that data and only then poll again.
I tried doing both I2CRequestSensorData() and processBuffer() in the main loop. Although it will catch all over-threshold values, there is a lot of latency when there are a lot of sensors over-threshold and the latency is unpredictable. It varies a lot from 1ms to 5ms. Although some latency is fine (and just a result of having to use a slow UART), I was hoping to get the UART to be more consistant by implementing the buffer.
please see my answer below. The device connected to the UART doesn't allow me to increase the speed. However, the average amount of data that should be sent is not that much. It's just that at some moments there's a lot of data available. All that data can't be pushed over the UART at the same time. That's why I though I could need a buffer so I can pile the data up in the buffer. After a small burst of data there is enough time to process it and send it over the UART. However, I want to keep "requesting" sensor data from the I2C peripherals. I don't want to wait until the pile of data is processed before returning back to the I2C peripherals. Hope this makes sense.
print() or write() will queue in the Serial buffer FIFO. You don't need another one if its depth is good enough for you.
As long as you don't fill up the Serial Tx FIFO buffer, your acquisition and printing out will work just fine and you'll get a steady output at the 9600 baud rate (assuming your I2C interrupts don't lock the system for too long)
I don't see what you call "latency" for the Serial line. if there are bytes in the FIFO buffer they will be emitted at the expected baud rate unless the Serial interruption to get the next byte cannot happen because you are busy reading the sensors and they lock the interruptions.
define "lots". if it amounts to more than 64 bytes (or possibly 256 bytes on your SAMD) needing to be pushed out, you'll exceed indeed the Serial Tx buffer and print/write becomes blocking.
You could also check available space in the Serial output buffer Serial.availableForWrite()
and do something if the buffer is near-to-full. What, well, that's up to you, but you could at least preempt blocking.
I'd be testing this all out at a high baud rate, sending it just to Serial Monitor, and playing with a few options. Much simpler that way.
OK - as long as you use the Arduino's SAMD core, you'll get the 256 byte deep Tx buffer then. You could also modify the source code (link in previous post) to increase the size of the buffer if really needed.
yes, as suggested in Post #3 you want to preempt the blocking of the Serial writes but then you need to decide what to do with the bytes...
whilst I don't think that's really needed and don't really like nor agree with the generalisation claims made by the author, you might want to look at the work done on bufferedOutput here
@fablableo Just some thoughts.
Your options are limited. You are gathering data at a rate beyond which you can transmit it in some circumstances, so, either,
A) your receiver will get data that is stale sometimes, not others. No big deal, if it's just being viewed by a human, but in a closed-loop control this could be quite deleterious.
B) your receiver will always receive up to date data, but some events will be suppressed. No big deal in most circumstances, unless the events represent process transients that MUST be known about (e.g. sudden transients into dangerous conditions).
This is why advising you how to deal with it is not easy. You know the end result, we don't.
For example, you could buffer the readings in a "single value & flag", transmitting whatever is in "single value" and clearing the flag.
Then, you'd have to decide if 'single value' holds the most recent conversion, the highest value since flag last read, etc, etc.
All up to you.
