Maximum speed that the Arduino can read an SD card

[dimavima]

Hi,

Someone can tell me what is the maximum speed that the Arduino can read an SD card?

I'm using the Arduino Mega 2560, Ethernet shield and a 2GB micro SD card.

With the sketch Analogger, I make only 10 samples.
And now I'm trying with another sketch read soon as possible.
With the function millis () I see it took 300 milliseconds.

It is possible to read more faster?

Greetings

[Grumpy_Mike]

You can read audio samples off an SD card at about 22KHz, so that translates into 45uS per 16 but sample. Perhaps it is the overhead of opening and closing the file that is consuming the time.

[dimavima]

hi,

Thanks for your answers.

I save the data in SD using the sketch AnalogLogger.
I just changed to make only 10 samples.
Inside the file:

Code: [Select]


millis,date,time,sens0,sens1,sens2
3600,2000/1/1,0:00:00,498,390,338
3700,2000/1/1,0:00:00,405,373,346
3800,2000/1/1,0:00:00,368,357,338
3900,2000/1/1,0:00:00,353,342,331
4000,2000/1/1,0:00:00,342,331,324
4100,2000/1/1,0:00:00,328,324,315
4200,2000/1/1,0:00:00,324,316,310
4300,2000/1/1,0:00:00,319,310,306
4400,2000/1/1,0:00:00,309,306,299
4500,2000/1/1,0:00:00,308,301,296


And use the following code to read the contents of the file.

Code: [Select]


const int chipSelect = 4;

#include <SdFat.h>
SdFat sd;
SdFile myFile;
uint8_t buf[450];

ArduinoOutStream cout(Serial);

void setup()
{
  Serial.begin(9600);

if (!sd.init(SPI_FULL_SPEED, chipSelect)) sd.initErrorHalt();

 
  if (!myFile.open("LOGGER00", O_READ)) {
    sd.errorHalt("opening test.txt for read failed");
  }

  int data;
  //while ((data = myFile.read()) > 0) Serial.write(data);
  Serial.println(millis());
  myFile.read(buf,340);
  cout << buf << endl;
  Serial.println(millis());
  myFile.close();
}

void loop()
{
}



I pulled the file extension to see if it would be faster but did not result.

Can you tell me how can I read faster or say what am I doing wrong?

Greetings

[billroy]

What happens when you time it without the cout?

-br
http://bitlash.net

[dimavima]

I put an attachment in my last post where the skecth I use to write on the SD.

[Grumpy_Mike]

Code: [Select]

Serial.println(millis());
  myFile.read(buf,340);
  cout << buf << endl;
  Serial.println(millis());
Is not going to tell you anything because of the time it takes to do a serial print.
Record the value of millis() in a variable, do the access. calculate the time by subtracting the recorded time before the access to the time now and then print out the result.

[dimavima]

Code: [Select]

Serial.println(millis());
  myFile.read(buf,340);
  cout << buf << endl;
  Serial.println(millis());
Is not going to tell you anything because of the time it takes to do a serial print.
Record the value of millis() in a variable, do the access. calculate the time by subtracting the recorded time before the access to the time now and then print out the result.

Thank you.
Now I see that it takes only 2 milliseconds to read the file and 11 milliseconds to open, read and close.

[fat16lib]

For file I/O, most SD cards can be read at about 250 KB/sec.  The class of card doesn't matter much for reading on the Arduino.

You can test you card by running the bench.pde sketch in the SdFat/examples folder.

The fastest possible read from an SD card on an Arduino is about 600 KB/sec.  This is for streaming multiple contiguous blocks using the SD multiple block read command.  SdFat does not use this command for file I/O since it has only a one block buffer.

Most of your 300 milliseconds is in the cout command, as billroy stated.

If you need to go fast, you must use binary data.  Formatting and parsing text on the Arduino take far longer than the SD I/O time.

I posted an example binary data logger that can log two-byte samples from a fast ADC at 40,000 samples per second without dropping samples.

The logger is here http://code.google.com/p/beta-lib/downloads/list.

More info is here http://forums.adafruit.com/viewtopic.php?f=31&t=21728.

[dimavima]

For file I/O, most SD cards can be read at about 250 KB/sec.  The class of card doesn't matter much for reading on the Arduino.

You can test you card by running the bench.pde sketch in the SdFat/examples folder.

The fastest possible read from an SD card on an Arduino is about 600 KB/sec.  This is for streaming multiple contiguous blocks using the SD multiple block read command.  SdFat does not use this command for file I/O since it has only a one block buffer.

Most of your 300 milliseconds is in the cout command, as billroy stated.

If you need to go fast, you must use binary data.  Formatting and parsing text on the Arduino take far longer than the SD I/O time.

I posted an example binary data logger that can log two-byte samples from a fast ADC at 40,000 samples per second without dropping samples.

The logger is here http://code.google.com/p/beta-lib/downloads/list.

More info is here http://forums.adafruit.com/viewtopic.php?f=31&t=21728.

Thanks for the tip. I will see and then I'll say one thing.

My goal was to read three analog inputs (10Hz) and send it by wireless.
If the connection failed I saved on the SD and later when the connection to enable sending the data that are on SD wirelessly without interrupting the reading.

[fat16lib]

10 Hz is a problematic speed for data logging to an SD. 

Common SD cards have occasional write latencies of over 100 milliseconds when used in file I/O mode on the Arduino.

Modern SD cards are not intended to be used in single block write mode but I must use this mode on the Arduino.

Rewriting file structures in this mode causes delays due erasing flash chips and moving data in the SD. 

SD cards now have 128 KB erase groups and some cards erase several groups since the spec allows up to 250 milliseconds write laency.

To log data reliably, not drop samples, at 10 Hz and above requires special techniques like in my fastLogger examples.

[dimavima]

10 Hz is a problematic speed for data logging to an SD. 

Common SD cards have occasional write latencies of over 100 milliseconds when used in file I/O mode on the Arduino.

Modern SD cards are not intended to be used in single block write mode but I must use this mode on the Arduino.

Rewriting file structures in this mode causes delays due erasing flash chips and moving data in the SD. 

SD cards now have 128 KB erase groups and some cards erase several groups since the spec allows up to 250 milliseconds write laency.

To log data reliably, not drop samples, at 10 Hz and above requires special techniques like in my fastLogger examples.

Hi,

Thanks for the explanation.
I've had to study fastLogger.pde and I understand more or less.

I need to measure three analog inputs and include the date and time of the RTC registers. In your opinion what is the easiest way to adapt the example?
Create 1 ring for each subject?

[oscarcar]

I'm not sure if the ethernet shield is in play here, but...

It would seem you only need to do 1 timestamp from the RTC at the first line.

After that, you just output the samples, since you know each sample will be .10 seconds after the last.

I plan to code a way to interlace the samples, along with an initial timestamp, but will be awhile before I get to it.

If I were you, I'd use fatlib16's example to write the binary data.
But in setup, you write out a separate file that just stores the initial timestamp in text.

Use his example python code to transcode the binary file into text on your PC.