Pages: [1]   Go Down
Author Topic: Maximum speed that the Arduino can read an SD card  (Read 4740 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi,

Someone can tell me what is the maximum speed that the Arduino can read an SD card?

I'm using the Arduino Mega 2560, Ethernet shield and a 2GB micro SD card.

With the sketch Analogger, I make only 10 samples.
And now I'm trying with another sketch read soon as possible.
With the function millis () I see it took 300 milliseconds.

 It is possible to read more faster?

Greetings
Logged

Manchester (England England)
Offline Offline
Brattain Member
*****
Karma: 597
Posts: 33314
Solder is electric glue
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

You can read audio samples off an SD card at about 22KHz, so that translates into 45uS per 16 but sample. Perhaps it is the overhead of opening and closing the file that is consuming the time.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

hi,

Thanks for your answers.

I save the data in SD using the sketch AnalogLogger.
I just changed to make only 10 samples.
Inside the file:
Code:
millis,date,time,sens0,sens1,sens2
3600,2000/1/1,0:00:00,498,390,338
3700,2000/1/1,0:00:00,405,373,346
3800,2000/1/1,0:00:00,368,357,338
3900,2000/1/1,0:00:00,353,342,331
4000,2000/1/1,0:00:00,342,331,324
4100,2000/1/1,0:00:00,328,324,315
4200,2000/1/1,0:00:00,324,316,310
4300,2000/1/1,0:00:00,319,310,306
4400,2000/1/1,0:00:00,309,306,299
4500,2000/1/1,0:00:00,308,301,296

And use the following code to read the contents of the file.

Code:
const int chipSelect = 4;

#include <SdFat.h>
SdFat sd;
SdFile myFile;
uint8_t buf[450];

ArduinoOutStream cout(Serial);

void setup()
{
  Serial.begin(9600);

 if (!sd.init(SPI_FULL_SPEED, chipSelect)) sd.initErrorHalt();

 
  if (!myFile.open("LOGGER00", O_READ)) {
    sd.errorHalt("opening test.txt for read failed");
  }
 
  int data;
  //while ((data = myFile.read()) > 0) Serial.write(data);
  Serial.println(millis());
  myFile.read(buf,340);
  cout << buf << endl;
  Serial.println(millis());
  myFile.close();
}

void loop()
{
}


I pulled the file extension to see if it would be faster but did not result.

Can you tell me how can I read faster or say what am I doing wrong?

Greetings

* AnalogLogger.pde (4.81 KB - downloaded 20 times.)
Logged

0
Offline Offline
God Member
*****
Karma: 39
Posts: 988
Get Bitlash: http://bitlash.net
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

What happens when you time it without the cout?

-br
http://bitlash.net
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I put an attachment in my last post where the skecth I use to write on the SD.
Logged

Manchester (England England)
Offline Offline
Brattain Member
*****
Karma: 597
Posts: 33314
Solder is electric glue
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Code:
Serial.println(millis());
  myFile.read(buf,340);
  cout << buf << endl;
  Serial.println(millis());
Is not going to tell you anything because of the time it takes to do a serial print.
Record the value of millis() in a variable, do the access. calculate the time by subtracting the recorded time before the access to the time now and then print out the result.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Code:
Serial.println(millis());
  myFile.read(buf,340);
  cout << buf << endl;
  Serial.println(millis());
Is not going to tell you anything because of the time it takes to do a serial print.
Record the value of millis() in a variable, do the access. calculate the time by subtracting the recorded time before the access to the time now and then print out the result.

Thank you.
 Now I see that it takes only 2 milliseconds to read the file and 11 milliseconds to open, read and close.
Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1601
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

For file I/O, most SD cards can be read at about 250 KB/sec.  The class of card doesn't matter much for reading on the Arduino.

You can test you card by running the bench.pde sketch in the SdFat/examples folder.

The fastest possible read from an SD card on an Arduino is about 600 KB/sec.  This is for streaming multiple contiguous blocks using the SD multiple block read command.  SdFat does not use this command for file I/O since it has only a one block buffer.

Most of your 300 milliseconds is in the cout command, as billroy stated.

If you need to go fast, you must use binary data.  Formatting and parsing text on the Arduino take far longer than the SD I/O time.

I posted an example binary data logger that can log two-byte samples from a fast ADC at 40,000 samples per second without dropping samples.

The logger is here http://code.google.com/p/beta-lib/downloads/list.

More info is here http://forums.adafruit.com/viewtopic.php?f=31&t=21728.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

For file I/O, most SD cards can be read at about 250 KB/sec.  The class of card doesn't matter much for reading on the Arduino.

You can test you card by running the bench.pde sketch in the SdFat/examples folder.

The fastest possible read from an SD card on an Arduino is about 600 KB/sec.  This is for streaming multiple contiguous blocks using the SD multiple block read command.  SdFat does not use this command for file I/O since it has only a one block buffer.

Most of your 300 milliseconds is in the cout command, as billroy stated.

If you need to go fast, you must use binary data.  Formatting and parsing text on the Arduino take far longer than the SD I/O time.

I posted an example binary data logger that can log two-byte samples from a fast ADC at 40,000 samples per second without dropping samples.

The logger is here http://code.google.com/p/beta-lib/downloads/list.

More info is here http://forums.adafruit.com/viewtopic.php?f=31&t=21728.

Thanks for the tip. I will see and then I'll say one thing.

 My goal was to read three analog inputs (10Hz) and send it by wireless.
 If the connection failed I saved on the SD and later when the connection to enable sending the data that are on SD wirelessly without interrupting the reading.
Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1601
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

10 Hz is a problematic speed for data logging to an SD. 

Common SD cards have occasional write latencies of over 100 milliseconds when used in file I/O mode on the Arduino.

Modern SD cards are not intended to be used in single block write mode but I must use this mode on the Arduino.

Rewriting file structures in this mode causes delays due erasing flash chips and moving data in the SD. 

SD cards now have 128 KB erase groups and some cards erase several groups since the spec allows up to 250 milliseconds write laency.

To log data reliably, not drop samples, at 10 Hz and above requires special techniques like in my fastLogger examples.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

10 Hz is a problematic speed for data logging to an SD. 

Common SD cards have occasional write latencies of over 100 milliseconds when used in file I/O mode on the Arduino.

Modern SD cards are not intended to be used in single block write mode but I must use this mode on the Arduino.

Rewriting file structures in this mode causes delays due erasing flash chips and moving data in the SD. 

SD cards now have 128 KB erase groups and some cards erase several groups since the spec allows up to 250 milliseconds write laency.

To log data reliably, not drop samples, at 10 Hz and above requires special techniques like in my fastLogger examples.

Hi,

Thanks for the explanation.
I've had to study fastLogger.pde and I understand more or less.

I need to measure three analog inputs and include the date and time of the RTC registers. In your opinion what is the easiest way to adapt the example?
Create 1 ring for each subject?
Logged

SF Bay Area
Offline Offline
Full Member
***
Karma: 1
Posts: 182
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I'm not sure if the ethernet shield is in play here, but...

It would seem you only need to do 1 timestamp from the RTC at the first line.

After that, you just output the samples, since you know each sample will be .10 seconds after the last.

I plan to code a way to interlace the samples, along with an initial timestamp, but will be awhile before I get to it.

If I were you, I'd use fatlib16's example to write the binary data.
But in setup, you write out a separate file that just stores the initial timestamp in text.

Use his example python code to transcode the binary file into text on your PC.
Logged

Pages: [1]   Go Up
Jump to: