next topic »

[on:]

Hello!

I have a data logging project where I am logging data from an MPU6050. It logs the data every 5 milliseconds. It is quite good and I have even implemented a real time clock. The problem is that it takes twoce as long every 15th reading. Like the following:

                                               Time between logs
928160   -1284   -1828   11188   -86   -40   -168   5004
933164   -1184   -1784   10988   -78   -51   -172   5004
938168   -1224   -1788   11044   -74   -37   -128   5004
943172   -1376   -1772   11012   -83   -52   -155   5004
948176   -1256   -1720   11024   -67   -18   -143   5004
953180   -1264   -1792   11136   -78   -68   -138   5008
958188   -1320   -1752   11044   -80   -46   -156   5004
963192   -1368   -1748   11068   -85   -44   -154   5008
968200   -1224   -1656   11096   -69   -20   -147   5000
973200   -1304   -1768   11072   -71   -59   -154   5004
978204   -1228   -1760   10968   -104   -34   -140   11392
989596   -1160   -1680   11148   -86   -21   -169   5004
994600   -1200   -1748   11108   -77   -35   -149   5000
999600   -1320   -1672   11096   -76   -42   -137   5004
1004604   -1328   -1712   11140   -88   -39   -162   5004
1009608   -1288   -1792   11068   -70   -37   -168   5004
1014612   -1232   -1732   11036   -83   -19   -141   5008
1019620   -1264   -1644   11064   -98   -46   -139   5004
1024624   -1332   -1764   11008   -83   -43   -138   5004
1029628   -1236   -1708   10964   -88   -36   -146   5008
1034636   -1264   -1728   11108   -94   -48   -133   5008
1039644   -1348   -1780   11076   -56   -37   -137   5004

I am logging with the following code:

      Serial.println(micros());
      accelgyro.getMotion6(&ax, &ay, &az, &gx, &gy, &gz);

      dataFile.print(micros());
      dataFile.print(",");
      dataFile.print(ax);
      dataFile.print(",");
      dataFile.print(ay);
      dataFile.print(",");
      dataFile.print(az);
      dataFile.print(",");
      dataFile.print(gx);
      dataFile.print(",");
      dataFile.print(gy);
      dataFile.print(",");
      dataFile.println(gz);

      Serial.println(micros());

I guess my question is: Any advice on how to make the arduino not drop these values. I have tried SDfat and sd and there is no difference. I guess the latency comes form the string conversion. Any ideas?

[Reply #1 on:]

The problem is that it takes twoce as long every 15th reading.

Not surprising. The earlier writes write to a buffer. The 15th one fills the buffer and requires actually writing to the card, which takes some time.

I guess the latency comes form the string conversion.

No, it doesn't, or all the writes would be the same speed.

Any ideas?

Don't log so often. Then, the commit part will complete before you need to write again.

[Reply #2 on:]

Thanks PaulS...

I built a string and then wrote it all at once. It actually made it worse. Maybe there is a better way.....See code below..

Code:

      accelgyro.getMotion6(&ax, &ay, &az, &gx, &gy, &gz);
          asis=String(ax)+","+String(ay)+","+String(az)+","+String(gx)+","+String(gy)+","+String(gz)+","+(micros())+","+("\r\n");
          y++;
     
      if (y=30)
      {
dataFile.print(asis);
y=0;
asis="";
    }
    Serial.print(asis);

[Reply #3 on:]

Write latency is inherent to SD cards.  The SD spec allows up to 200 milliseconds latency for a write.  Occasional write latencies of 100 milliseconds are not uncommon.

You must design your logger to allow for latency.  I have posted several example fast loggers.

The cleanest way is to use a RTOS with two threads and read sensors in a high priority thread and buffer the data to be written to the SD in a lower priority thread.

I posted RTOS examples as ChibiOS20120529.zip and FreeRTOS20111031.zip here http://code.google.com/p/beta-lib/downloads/list.

Another way is to read the sensor in an ISR and buffer the data for loop.  AnalogIsrLogger20120810.zip posted in the above location is a very fast logger that reads the Arduino ADC in an ISR and write the the data in loop().

Several other demo sketches are in fastLoggerBeta20110802.zip.  This file is now a bit old.

[Reply #4 on:]

http://elm-chan.org/docs/mmc/mmc_e.html

Find 'Improving Write Performance' and see the diagram showing multiple sector write vs single sector write. Each time a block of NAND flash is to be written to, it first gets erased and then you write 1 or many sectors before the next erase. You get 2:1, at least your program is being efficient.

The size of the card can make a difference. Small cards, like 128M have smaller blocks to erase.

[Reply #5 on:]

Chan's website is very good but most of the stuff about performance of SD cards is out of date. 

Wear leveling in modern TLC flash can dominate latency.  Modern cards move data that has not been changed for a long time to a new physical location.  Flash controllers have various amounts of RAM buffering and use different algorithms so generalizations like small cards are faster just isn't true.

Write latency also depends on the access pattern.  328 Arduinos have limited memory so there are not many caching options. 

Small cards are often formatted with small clusters which increases the overhead since the FAT must be accessed often.

For embedded applications cards with SLC flash tends to be best.  Modern consumer cards use MLC or TLC flash.  Industrial SD cards mostly use SLC flash.  I have had good luck with this cheap Industrial card http://www.newegg.com/Product/Product.aspx?Item=9SIA12K0CT6829.

No card has uniform low write latency.  All cards have occasional longer latencies.  You must design fast data loggers with this in mind to avoid data loss.

I am now working on improved performance for SdFat with Cortex chips and the Mega.  Here is a link to some results for the Due with DMA SPI http://arduino.cc/forum/index.php/topic,134512.0.html.