I2C Help, Reading Buffers Correctly

Projects Networking, Protocols, and Devices
1

Hello,

This is my first post so plz go easy on me.

A quick foreword,
I am trying to to best understand I2C protocol and really use it to its maximum potential and not build bad habits. I have an LSM9DS1 orientation sensor that I am using for an IoT project. I have successfully used the sensor using node, then python, now C++. I see that the lower level languages seem to be better for this kind of work bc they have more control and use less resources (I think). One thing I am noticed is that the readings can be pretty noisy (I think is the right term) so averaging can really help reduce the noise. Naturally, I came to the conclusion that the faster I can read data, the more I can average the data, the more accurate the data is. So, I read the datasheet for the sensor more carefully and some of the concepts are hard to understand as they are constantly jumping from register to register. It can be a bit confusing to get the full picture. What I am focused on in this post is:

"The LSM9DS1 embeds 32 slots of 16-bit data FIFO for each of the gyroscope’s three output channels, yaw, pitch and roll, and 16-bit data FIFO for each of the accelerometer’s three output channels, X, Y and Z."

And more specifically I just want the Accelerometer buffer for now at least.

The Questions:

  • If I am using the FIFO and interrupt pin, what is the best way to burst read data using I2C?
  • If one I2C message can carry 2 bytes (16 bits, 1 word), Can I just read the "low" register and I2C will give me the 16 bit word, low and high together?
  • (Continuation of 1.) If I am attempting to empty the FIFO with a burst read, do I read:

In code it would look something like:

readRegisters(foo, OUT_X_XL, 32)
readRegisters(foo, OUT_Y_XL, 32)
readRegisters(foo, OUT_Z_XL, 32)

VS

for (32)
readRegister(foo, OUT_X_XL)
readRegister(foo, OUT_Y_XL)
readRegister(foo, OUT_Z_XL)

Afterword,
My thinking is if I can utilize the built in buffer and the interrupt pin, then I can free up the i2c bus and all the arduino needs to do is average the data. Basically, wait for the FIFO to fill, burst read the data, then average the read data. I am sure this is a common question, but whenever I research I2C, there seems to be plenty on doing basic I2C reading but not a whole lot on the proper way to utilize the FIFO buffer as the manufacturer intended.
Hopefully someone can help! Thanks in advance!

2

Which Arduino board do you have ?
Can you give a link to where you bought your LSM9DS1 module ?

If you can use the SPI bus, then you don't have to deal with the slow I2C bus.

The basic Arduino boards (such as the Arduino Uno) have a buffer of 32 bytes for the I2C bus.
The FIFO of the sensor has 32 slots of 16 bits, that makes 64 bytes. That means if the FIFO buffer is 3/4 filled with data, you need two I2C sessions with the Wire library if you have a basic Arduino board.

The FIFO of a sensor is often used as this:

volatile bool newData;

void setup()
{
  attachInterrupt( digitalPinToInterrupt( ...), fifoInterrupt, ...);
}

void loop()
{
  if( newData)
  {
    newData = false;

    Wire.requestFrom( ...
  }
}

void fifoInterrupt()
{
  newData = true;
}

The interrupt sets a variable and the loop() processes it.
How the data is organized in the FIFO, that should be in the datasheet.
The Wire.requestFrom() can retrieve a number of bytes, you have to convert that to 16-bit integers with the accelerator value.

3

Thanks so much for the reply Koepel! I have the newest UNO and here is the link to the sensor(board?) I am using.

Is SPI a better(more mature?) protocol?

Wow, I had no idea the Uno had a buffer as well. Im assuming the buffers need to be equal size to have a solid data stream. Would it be better to limit the FIFO buffer on the sensor to 32bytes (I can do this by setting the watermark value).
Or, would it be better to keep the FIFO full and use 2 burst read sessions?

My code looks very similar to yours. My ISR is to read the buffer into some XYZ global variables, my loop is just printing some global XYZ variables every .1 seconds.

global x, y, z
setup() {
set up LSM9DS1 object i made
set up interrrupt --> isr=readBuffer
}

readBuffer() {
readAccelFIFO(x, y, z)
}

loop() {
print(x, y, z)
wait(.1 sec)
}

Really appreciate the help!

4

What is the newest Uno ?
That is a 5V board (with a 5V I2C bus).

The module from Adafruit is okay. It is compatible with 3.3V boards and with 5V boards.
There is a link to a tutorial that show how to use the hardware SPI bus in the sketch.
You could try the Adafruit library with the SPI bus first before turning on the FIFO.

You should not run a library function that is interrupt-driven from inside an interrupt routine.
That is where that variable is for.
The larger the buffer, the better. You could do an extra check to see if the FIFO is empty.

The SPI bus has strong signals and it is fast and the code (of the library) is small.
The I2C bus has weak signals and it is slow. It uses interrupts and it needs a lot of code (in the library).

5

I am using the Arduino Uno Rev3.

Looks like I need to do some SPI research and what not. I will also check out Adafruits SPI library.

So if SPI is so quick would the proper way(industry way) be to just read live data at max speed and forget about the FIFO all together? Or, would the proper way be to utilize the FIFO with SPI to limit the amount of messages sent and not read the same data element twice before the sensor has a chance to update?

The max ODR for the accelerometer is 952 Hz. Is that I2C speed or SPI speed? Im not sure how Hz translates to kbps.

Thank you so much Koepel, you are helping from shooting in the dark here.

6

Manufacturer's page of the LSM9DS1 : https://www.st.com/en/mems-and-sensors/lsm9ds1.html with datasheet.

ODR = Output Data Rate.

Good question if the ODR of 952 is for I2C or SPI. I can not find that information in the datasheet.
The I2C bus can be 400kHz, 9 clock pulses per byte, 1 clock pulse overhead. 3 channels of 2 byte is 6 byte. That makes 400khz / (9+1) / 6 = 6.7kbyte 6700 possible samples per second. Enough to keep up with the highest setting in the sensor.

To do mathematical calculations, the samples has to be taken at a fixed interval.
If a processor can do good timing with precise intervals and uses SPI, then the FIFO is not needed.
If a processor can be busy with other things and uses I2C, then the FIFO might be handy.
They made a sensor that is versatile and can be used in different ways.

7

Is this all about a high sample rate to be able to use the average ?

A accelerometer is noisy, a gyro drifts and a magnetometer is disturbed by other things. Every bad side of each sensor can be compensated by the other sensors. That is called "sensor fusion". There are mathematical ways to combine them. For example a Kalman filter or a AHRS filter. The result of that filter is more than just the three sensors.

8

Koepel,
So I recently received my BS in Computer Science and I sadly didn't have the opportunity to work with hardware besides one of those silly parallax maze navigating robots. That being said I've been personally learning about sensors and IoT in my spare time because I find it more fun than WebDev and what not. Naturally, I am would love to hold a job in the future where I get to work with sensors and things that interact with the physical world. So I've been trying to learn the right way to do it. The industry standard way if possible. I plan to work through the different protocols all the way up to CAN & USB hopefully.

Like I mentioned earlier. I have a pretty stable and fast I2C data stream going, but I would like to learn the most correct way to interface with this type of sensor. I figure if I learn this sensor correctly then I can learn sensors in the future more easily. I'm doing a deep dive so to speak.

"The I2C bus can be 400kHz, 9 clock pulses per byte, 1 clock pulse overhead. 3 channels of 2 byte is 6 byte. That makes 400khz / (9+1) / 6 = 6.7kbyte per second."

Can you elaborate on:

"9 clock pulses per byte, 1 clock pulse overhead. 3 channels of 2 byte is 6 byte"

So does the 9 pulses per byte come from the 8 bits per byte + the ACK bit?
1 pulse overhead is the stop bit?

Every bad side of each sensor can be compensated by the other sensors. That is called "sensor fusion"

This is very interesting. It makes sense. Would this be a good example?

  • You detect a change in heading (0-360) from the Mag but you do not detect a change in the Gyro. The Gyroscope would have detected a rotation if the heading had changed.

Koepel, you seemed to have a great knowledge on this work. How did you learn about this? School? Books? Mentor?

Can you recommend any resources that might best help me? I am a "simple man" and anything too "textbook" will put me right to sleep :sweat_smile:

Thanks again by the way.

9

Hey! I did it! (The I2C FIFO)

If you are at all interested this is the code I listed it below. I started with the slowest ODR of 10hz, got it working, then ramped it up to max ODR 952hz. I also listed some output. Seems I can burst read 32 samples of each X,Y,Z, and average them, every 0.035 seconds which seems pretty good. Although I do have some suspicions that I still have something messed up in my ctrl registers (write registers).

#include <LSM9DS1_FIFO.h>

bool fifo_is_full = false;
long int start_time, end_time;
int current_fifo;

long int i = 0;
float x, y, z;

byte interruptPin = 2;

TwoWire bus;
LSM9DS1_FIFO sensor(bus);

void setup() {
  Serial.begin(9600);
  while (!Serial);
  Serial.println("Started");
  
  sensor.setGyroAccelAddress(0x6B);
  sensor.setMagAddress(0x1E);
  sensor.setBufferSize(32);

  if(sensor.begin()) {
    if(sensor.useFIFO()) {
      pinMode(interruptPin, INPUT_PULLUP);
      attachInterrupt(digitalPinToInterrupt(interruptPin), read, RISING);
      start_time = millis();
      Serial.println("READY");
    } else {
      Serial.println("FAILED: useFIFO()");
    }
  } else {
    Serial.println("FAILED: begin()");
  }
}

void loop() {
  if(fifo_is_full) {
    sensor.readAccelBuffer(x, y, z);
    fifo_is_full = false;
    print();
  }
}

void print() {
 end_time = millis();
  
  Serial.print("Count: ");
  Serial.print(i);
  
  Serial.print("\tTime: ");
  Serial.print((end_time - start_time)/1000.0, 4);
  
  Serial.print("\tX: ");
  Serial.print(x);
  
  Serial.print("\tY: ");
  Serial.print(y);
  
  Serial.print("\tZ: ");
  Serial.println(z);
  
  i += 1;
  start_time = end_time;  
}

void read() {
  fifo_is_full = true;
}

Here is some output at 952hz

(who am i) check SUCCESS
READY
Count: 0	Time: 0.0640	X: -0.69	Y: -0.63	Z: -0.47
Count: 1	Time: 0.0340	X: -0.69	Y: -0.63	Z: -0.47
Count: 2	Time: 0.0350	X: -0.69	Y: -0.63	Z: -0.46
Count: 3	Time: 0.0340	X: -0.69	Y: -0.63	Z: -0.46
Count: 4	Time: 0.0340	X: -0.69	Y: -0.63	Z: -0.47
Count: 5	Time: 0.0340	X: -0.69	Y: -0.63	Z: -0.47
Count: 6	Time: 0.0350	X: -0.69	Y: -0.63	Z: -0.46
Count: 7	Time: 0.0340	X: -0.69	Y: -0.63	Z: -0.47
Count: 8	Time: 0.0350	X: -0.69	Y: -0.63	Z: -0.46
Count: 9	Time: 0.0330	X: -0.69	Y: -0.63	Z: -0.47
Count: 10	Time: 0.0350	X: -0.69	Y: -0.63	Z: -0.47
1 Like
10

Since fifo_is_full is being used in multiple routines and can change at any time (due to interupt) it should really be declared as volatile. You might get away with it in your current code but any alteration could cause the code to stop working and you might wonder what has caused it. This typically happens when the compiler optimises the code and keeps the value in a register, the interupt changes the location in memory but the optimised code only looks at the register.

11

Good to know countrypaul!

Do you think it would be a good idea to make my float x, y, z volatile as well since they are accessed so frequently or would allowing x,y,z to be stored in a temp register actually benefit the programs speed/efficiency?

My operating systems class was pretty dry + covid really put a wrench in it.

If I am using float x, y, z as global variable then the compiler will put it in the stack. Every time I pass a reference of x,y,z to readAccelBuffer I am effectively just changing the value of the variable in the stack, not deleting and reinitializing the variable further down the stack. The stack is more quick than the heap so the way I have it now would be superior to using something like malloc. That being said, if I declare the variables volatile then I am forcing my statement "if(is_fifo_full)" to look up is_fifo_full on the stack and not risk is_fifo_full being stored in a temporary register. if is_fifo_full was pushed to a temporary register then it could cause some really buggy behavior that would be very hard to debug.

Does all of that sound somewhat right?

Also I still don't quite understand why we do this...
"The interrupt sets a variable and the loop() processes it"
It works great but why not just have the ISR do the buffer read?

12

volatile directs the compiler to not optimise the variable into a register and always access it in memory - therefore it slows things down slightly. So if your variables are not likely to change in an interupt there is no advantage in making them volatile and potentially a significant performance hit in doing so.

If I am using float x, y, z as global variable then the compiler will put it in the stack. Every time I pass a reference of x,y,z to readAccelBuffer I am effectively just changing the value of the variable in the stack, not deleting and reinitializing the variable further down the stack. The stack is more quick than the heap so the way I have it now would be superior to using something like malloc. That being said, if I declare the variables volatile then I am forcing my statement "if(is_fifo_full)" to look up is_fifo_full on the stack and not risk is_fifo_full being stored in a temporary register. if is_fifo_full was pushed to a temporary register then it could cause some really buggy behavior that would be very hard to debug.

Most of what you say about the stack and heap appears correct, but remember they are the same memory and excessive use of one can cause it to collide with the other - again difficult to work out what has gone wrong when this happens. Generally using malloc and free on the Arduino can lead to memory fragmentation as there is no garbage collection scheme.

Also I still don't quite understand why we do this...
"The interrupt sets a variable and the loop() processes it"
It works great but why not just have the ISR do the buffer read?

On he Arduino whilst processing an interupt other interupts are disabled (unless specifically enabled). I assume that reading the buffer normally triggers an Interupt so that would not work within an ISR.

13

putabirdyonit:
Can you elaborate on:
The I2C bus can be 400kHz, 9 clock pulses per byte, 1 clock pulse overhead. 3 channels of 2 byte is 6 byte. That makes 400khz / (9+1) / 6 = 6.7kbyte per second.
So does the 9 pulses per byte come from the 8 bits per byte + the ACK bit?
1 pulse overhead is the stop bit?

I think I wrote that wrong :-[ It is not 6.7 kbyte per second, but 6.7 kHz sample rate (each is 3 samples and each sample is 2 bytes).

Yes, 8 bits + ACK bit makes 9 clock pulses. The I2C Master sends out 9 clock pulses for every byte.
The 1 clock pulse for overhead is some extra time for the overhead. To make the calculation easy. I assume that the extra time for the overhead is equal to the time of 1 clock pulse of the I2C bus. That is silly, they are not related. There is no 10th clock pulse. But it makes the calculation easy :wink:

putabirdyonit:
Hey! I did it! (The I2C FIFO)

Very cool :smiley:

I suggest to learn about the AHRS filter. It is worth your time.
This is what it is: Attitude and heading reference system - Wikipedia.
There is a old Adafruit tutorial and a new Adafruit tutorial.

14

I will be sure to look at those links Koepel.
Also thank you for the insight coutrypaul. I think it makes sense that the actual I2C read is probably an interrupt itself.

I think I celebrated too soon :slight_smile:

So the board has multiple FIFO modes. In the most basic mode, the FIFO will stop when full. The mode I will use is when the FIFO is full, the oldest sample will be pushed out (just like a FIFO data structure). However I noticed that by the time I have finished reading the FIFO it has already started filling back up. I'm not sure why but I assume it has to do with timing and how long it takes to send ~200 bytes of data through I2C (slave --> master). The actual C++ code might be contributing to some lag as well. Here is an example:

@ 115200 baud

ODR = 120hz --> 3 rows already back in FIFO

FIFO Buffer: 30
Count: 1	Time: 0.5390	X: -0.38	Y: -1.72	Z: -1.17
FIFO Buffer: 3

ODR = 240hz --> 7 rows already back in FIFO

FIFO Buffer: 27
FIFO Buffer: 159
Count: 2	Time: 0.1420	X: -0.37	Y: -1.72	Z: -1.17
FIFO Buffer: 7

ODR > 240hz adds some really weird behavior. For some reason the buffer is read twice.

FIFO Buffer: 30
FIFO Buffer: 159
Count: 2	Time: 0.0980	X: -0.36	Y: -1.72	Z: -1.17
Count: 3	Time: 0.0340	X: -0.36	Y: -1.72	Z: -1.17
FIFO Buffer: 1
FIFO Buffer: 1

OR

FIFO Buffer: 30
Count: 0	Time: 0.0970	X: -0.34	Y: -1.72	Z: -1.17
FIFO Buffer: 15

At this point I think I would need a logic analyzer and oscilloscope to watch the data lines and figure out what is happening (which I don't have the budget for right now).

That being said, I think the next thing I will be doing is trying the same idea but with SPI. Maybe the increase in speed will rid the buggy behavior altogether. That is to say, maybe I can read the ~200 bytes in less time and therefore the FIFO won't have the opportunity to refill.

I may also try using the FIFO where it stops once full. This would allow me to use the SPI write line to empty the FIFO after the buffer read without clogging the data line bc they are on different wires. (is my thinking)

15

I noticed that the name of the variable is "fifo_is_full", but I thought that was just a wrong name. No one waits until a FIFO is full.
Generate an interrupt as soon as there is data in the FIFO, no matter if it is only a few bytes. That is how the Arduino can keep up, and the FIFO will be a "buffer" not a "fill up queue". After all, a FIFO is supposed to be a buffer after all.

When the Arduino does calculations, then the FIFO might be filled a little more, but it should not get full.

You can give your variable an other name :wink:

volatile bool helloThereIsSomethingInTheFifoSoYouBetterHurryToReadIt

I'm very fond of the 30 dollar LHT00SU1 with sigrok / PulseView. Turn off the analog channel in PulseView to get a higher sample rate for the digital inputs. In Windows you need a program "zadig" to install a driver. Instructions are on the sigrok website. PulseView can decode the I2C signal, so you can see the data.
By the way, the 10 dollar 8 channel 24MHz logic analyzer will also work, but I think it is mechanical too cheap and the usb connector does not make good contact or breaks.

16

Koepel,

Wow, this is all news to me. I just figured you fill the FIFO and then burst read all of the samples to be "efficient". It does make sense that that the FIFO is there so that "no sample is left behind".

Let me ask you this.
How would you set this up professionally?

Some of my thoughts...

  • I could set the interrupt to 0 or 1 sample and then just keep reading sample rows until the interrupt "turns off" (goes low). That way if the FIFO is > 1, I can make sure it is emptied.
  • I could set the time I am looking to get updates and then read and average what is available. Let's say I want a sample every .01 seconds. I could set a .01 seconds interval and then read the FIFO_SRC which will tell me how full the FIFO is and then I can read that number of rows and average them.
  • I could buy the logic analyzer and analyze just how busy the data line is.
  • I can experiment with different I2C clocks. For example, 400000 Hz (fast mode)
  • Use SPI not I2C because it is much faster than I2C.

By the way, you and countrypaul have been extremely helpful. I was worried this forum was going to be really harsh like StackOverflow. Thank you for being understanding, helpful and nice while you do it. I am doing this project because it really does interest me. It's nice to see/meet other people who are interested by it too!

17

Are you afraid that the I2C bus wears out ?
I don't care how busy it is. The I2C bus should be 100% reliable in the first place.

The best option is that when there is one or more samples, then retrieve it until the FIFO is empty.
That way, there is a buffer (the FIFO) that buffers data when the Arduino is doing other things. That is how it is supposed to work.

Of course, when tuning a system, it might be better to reduce the interrupts and reduce the slow I2C functions and wait until a few more samples are in the FIFO. That is something for later and hopefully it is not needed.

A 400kHz clock is normal when the wires are short.

The SPI bus is faster and better and stronger. It has major advantages.
There are however a few disadvantages:

  • Voltage levels must match (no fooling around with voltage levels as can be done with the I2C bus).
  • Sometimes a SPI device does not release the MISO line when it is not selected with the ChipSelect. In that case no other SPI devices can be added.

Compare that with the I2C bus with the Wire library:

  • Slow. The 400kHz helps, but I consider that still as slow for any Arduino board that is faster than a Arduino Uno.
  • Blocking. The Wire library is blocking, it is literally waiting for something to happen on the I2C bus. All that CPU time is wasted.
  • Weak. The high level is made with pullup resistors. Therefor the high level is vulnerable for noise. The voltage rises slowly and hopefully reaches a high level. It is like hoping that sun will come out after a rainy day.
  • Wrong. Many use the I2C bus and the Wire library wrong because they think of it as a "bus" that can carry data somewhere else, then I have to say: "It is not that kind of bus".

If I write more, then I start repeating myself: SPI is better than I2C; very fond of LHT00SU1; AHRS; Read data immediately from FIFO as soon as there is something; How to make a reliable I2C bus;

18

Got it! Thanks Koepel!

19

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.