I have a setup where every 2 milliseconds, 24 bytes of data needs to be continuously written to a file on the SD card. Unfortunately this needs to take a maximum of 800 us as the other 1200 us are spent on collecting and processing the data before it is written to the card. This sample-save-repeat routine could go on for a period of up to 5 minutes (24 bytes x 50 x 60 x 5 = 360 KB in 5 minutes). Using the sdfatlib, what is the most efficient way of going about this? I'm assuming buffering and block size are important, although I have very little knowledge in this area.
I also have a similar situation where the 24 bytes of data needs to be continuously read from a file on the SD card every 2 milliseconds. Preferably this read would take ~800us as I need ~1200us to do processing with it. What's the best way to do this?
I have also been looking into SPI serial flash from Microchip - I would save the data on the flash chip and then transfer the data to the SD card after a "capture session".
Special techniques are required to log data reliably at rates faster than about 5 samples per second. This is because SD cards can have occasional block write times of about 200 milliseconds.
The specification for SD cards assumes the host will have large buffers and data is written in very large blocks in 4-bit SD mode with a very fast clock. The specification for a class 10 card assumes a 40 MHz clock, 4-bit bus, and a recording block size of 512 KB. The speed is averaged over a 4MB write.
It is possible to record data at high rates but it is more complex than at slower rates since you need to capture data in an interrupt routine or use a RTOS and provide sufficient buffering.
Thanks for the help. I'm going to look at the fastLogger that you've made. As far as my second question about reading at a high speed - can that be achieved?
You will not always be able to read in less than 2 ms. To read a single 512 byte block from the SD takes about 900 usec . Some reads require accessing the FAT and other files structures so the time is longer. Each access reads a 512 byte block.
You can run the SdFat bench example to see how your SD performs. Here is typical output:
Type any character to start
Free RAM: 1045
Type is FAT16
File size 5MB
Starting write test. Please wait up to a minute
Write 178.48 KB/sec
Maximum latency: 148180 usec, Avg Latency: 555 usec
Starting read test. Please wait up to a minute
Read 288.45 KB/sec
Maximum latency: 2860 usec, Avg Latency: 342 usec