Best performance is indeed made with buffers equal to the sector size of the SD card (512 bytes).
You say one record consists of 14 bytes, [millis, f1,f2] comma separated fields. But how do you separate records?
Consider writing pure binary: (the commas do not add anything except 2/14 slack ~14%.
4 bytes long millis
4 bytes float f1
4 bytes float f2
So every 12 bytes is one record. 512 bytes can hold 512/12 = 42 records, leaving 8 bytes .
So in your main loop() you should have a counter that writes the 512 bytes buffer to SD after 42 records.
while writing this buffer you should fill buffer 2. etc
If you think reading 12 bytes records is difficult you could consider writing 16 bytes records of which exactly 32 fit in 512 bytes.
Room for an additional sensor 
Hello, thanks for your reply!
I should explain what the data is for. My Arduino is serving a web page, and I'd like to have a live graph that plots the two values (the two floats) with time. I intend to use a JavaScript library called DyGraphs, which can use arrays or comma separated values to display multiple series, which is why the commas are in there for now.
So will each entry to the buffer contain the string with all 3 values at that time? Or just one long array? I tried to code it, but I didn't make much progress, I can't think straight right now...
P.S. This isn't strictly relevant, but... I was concerned that the script would not be able to read as the file is being written to, but I think with the right update rate that should not be a problem. I have a feeling I will need to overwrite existing data after a soft reset (e.g. when the page is reloaded) so that the graph doesn't contain old, irrelevant data, but I will sort that out later.
To log 1000 samples per second requires a totally different technique than your sketch.
I suggest you look at example sketches in fastLoggerBeta20110802.zip here http://code.google.com/p/beta-lib/downloads/list.
Forget advice like 512 byte buffers unless you are doing raw writes to the SD.
The big problem is that a write to a file on an SD can take 200 milliseconds. You must capture data while the SD write is in progress. This means interrupts, probably using timers, and cleaver buffering schemes.
Thanks for your reply!
Will I be able to use interrupts if I have other things going on in my sketch? Could I remove the code related to the ADC chip?
As I will be saving the data for a graph on a webpage, I can't save the data in binary format, and I will have to store it somewhere; I assumed a file on the SD card would be the 'best' place. I'm considering increasing my interval too, 1000 times a second for a graph seems unnecessary. I tried writing some code based on the SPI EEPROM example, but it went horribly wrong!
#include <SD.h>
char buffer[512]; // For storing data
int i = 0; // count
int r = 1234; // "reference"
int delta = 10; // sampling interval, ms
long previousMillis = 0;
const int chipSelect = 4;
const int yPin = A5; // motor output, after scaling
void setup() {
Serial.begin(115200);
Serial.print("Initialising SD card...");
pinMode(10, OUTPUT); // CS pin
pinMode(yPin, INPUT);
// see if the card is present and can be initialised:
if (!SD.begin(chipSelect)) {
Serial.println("Card failed, or not present");
return;
}
Serial.println("card initialised.");
}
void loop() {
File dataFile = SD.open("graphlog.txt", FILE_WRITE);
unsigned long currentMillis = millis();
int y = analogRead(yPin);
String graphString = String(millis());
graphString += ",";
graphString += String(r);
graphString += ",";
graphString += String(y);
if(currentMillis - previousMillis == delta) {
previousMillis = currentMillis;
for (i=0; i < 510; i++) { // 512?
buffer[i] = graphString;
}
}
}
void fill_buffer() {
for (int i=0; i < 512; i++) {
buffer[i] = i;
}
}Edit: I started typing the above when there were only 2 replies!
If you do a 1000 samples per second, why add the millis() timestamp?
If evenly divided you just get consecutive numbers, if not you get multiple samples with the same timestamp (implying a zero duration between). using the micros as timestamp gives at least an - sub milli - interval between two samples.
I haven't found a way to specify the time between data points using DyGraphs, unfortunately, so I need to add the time so the script knows what to plot! I'm starting to think a much much lower sample rate would be more reasonable... this is all much more complicated than I anticipated!
For raw writes you must write 512 bytes.
For file writes there is some gain if all writes are 512 bytes and aligned on 512 byte boundaries.
For most apps this gain is not worth the effort. My benchmark example in SdFat uses a 100 byte buffer.
With a SanDisk 1 GB Ultra II card formatted FAT16 with the SdFormatter example I get this result:
This is a very good card for data logging. Not because of the average write rate of 198 KB/sec but the max latency of 81396 usec. Some cards have a max write latency of 200000 usec.
The 198 KB/sec is way faster than needed to log 1000 records per second with 14 byte records.
Once again the design problem is to overcome the occasional long write latency that is inherent in SD cards.
Even class 10 cards have this problem. The assumption is that devices like video cameras have lots of buffer so they achieve high average write rates for very large files. This allows occasional pauses by the card to erase large areas of flash. You can only write to erased flash.
I just ran bench with my little 256 MB card, formatted using SDFormatter. Here are the results:
Type any character to start
Free RAM: 1043
Type is FAT16
File size 5MB
Starting write test. Please wait up to a minute
Write 165.30 KB/sec
Maximum latency: 107708 usec, Avg Latency: 600 usec
Starting read test. Please wait up to a minute
Read 325.54 KB/sec
Maximum latency: 2336 usec, Avg Latency: 302 usecBy file write do you mean writing to a file (such as a CSV file)? If I don't need to have a 512 byte buffer, then would it just need to be long enough to store data points for more than the length of the maximum latency?
