Fast SD Card Data logging

Using Arduino Storage
1

Hi All,

I need someone to point me in the right direction.

I am logging data from an accelerometer and a gyro to an SD Card. My goal is to sample data with a rate of 500 samples/second. Each sample is comprised of 7 2 byte values, so 14 bytes in total. When writing to the card, it translates to about 40 bytes in ASCII format.

I started with the Datalogger example in Arduino which uses the SD library. So far, I can't even log data at 100 samples/second as the data loss is around 30% at this rate.

This probably means that the process of writing to the card is really slow. I found this
post about data flushing and used this method but it doesn't result in any improvement.

What can I do to speed things up? I have heard about people using SDFat library instead but I am not sure.

The SD card I am using is a class 10. How can I know if the SPI interface has been setup for fast speed. I don't see any SPI settings in the SD library.

My writing loop for reference:

loop(void)
{

/Code to read data from sensors goes here/

String dataString = "";
dataString+=String(millisecond);
dataString += ",";
dataString+=String(accel.x());
dataString += ",";
dataString+=String(accel.y());
dataString += ",";
dataString+=String(accel.z());
dataString += ",";
dataString+=String(gyro.x());
dataString += ",";
dataString+=String(gyro.y());
dataString += ",";
dataString+=String(gyro.z());

File dataFile = SD.open("datalog.txt",O_CREAT | O_WRITE);
if (dataFile) {
dataFile.println(dataString);
dataFile.flush();
dataFile.close();
}

2

Use write.
It can be like this

#include <SPI.h>
#include <SD.h>
File dataFile;
union 
{ char dta[16];
  struct  {
   long my_time;
   int ac_x, ac_y, ac_z, gy_x, gy_y, gy_z;
  };
} my_data;
//------------- then
void setup() {}
void loop() 
{
  my_data.my_time=millis();
  // my_data.ac_x=accel.x(); // lines like theese
  // 5 more of theese
  // open logfile -> I'd use a counter/timer to open/close file epproc every 10 sec
  dataFile = SD.open("datalog.dta"); // now not a textfile
  // print all data to file
  for (byte i=0;i<16;i++) dataFile.write(my_data.dta[i]);
  dataFile.close();
}

or you can write witout storing values first:
dataFile.write(highByte(ac_x=accel.x()));
dataFile.write(lowByte(ac_x=accel.x()));

3

Thanks for the answer.

This way I'll be storing binary values. right? I need to read these values on my computer once the logging has been done. How can I do this?

Can you explain the timed opening/closing part a bit? Does the loop operate without closing the file while in the 10 second period and keep writing? Does the file has to be closed before reaching the 512 KB buffer limit?

Thanks

4

It this is logging fast (many times a sec), there is no need to open/close the file during the logging.
Close when logging is done.

But if: logging is once an hour ? Then the data will be written to the card (512bytes/16bytes) every 32 hours.
You dont want a powerdown after 30 hours (all data lost)
Hope you can see the 'problem'.
If you want writing to happen enery n hours or logging point: count or use millis() to determine when to flush data to the card. (all 512 bytes in the buffer are written when a file is flushes or closed)

Export to a PC (excel?)

  1. Write a short routine that opens and reads the records and writes as comma separated values to a second file.
  2. Let Arduino read the record and write them to the serial port. A simple Terminalprogram e.g. "tera term" can recieve them an write them to a file on your PC (or copy /paste??)