Hi there, apologies if this has been asked before but I’m looking for some clarity on the SdFat 512 byte buffer.
I have a data logger storing sensor readings as float types. To minimise power consumption and preserve the sd card I have a loop storing the readings in an array before writing to the sd card every few seconds rather than every time I take a reading. Is this actually necessary? If the buffer is only written to the sd when it’s full that means 512/12 = 42 readings (float and comma separator). Does that make sense? Also can I increase the buffer? How does lowPower sleep affect the buffer?
you don't need your own buffer, you can rely on the built-in buffer to be dumped when needed. Changing its size would mean modifying the library and you still have the SD at the back of this that has specific needs for the buffer size anyway.
Of course relying on the library to actually perform the dump onto the SD card means you are at risk of loosing up to 512 bytes of data in case of power loss ➜ so you might want to decide to save more often
What would you win from having your own larger buffer ?
on an AVR like UNO or MEGA, the memory is kept intact.
I am using the ExFatFormatter to store my sensor readings. 6 floats = 24bytes which has to be padded to 32 bytes as per the EFF code. You also need to 'format' the SD card using the utility that is part of exFat. The data is stored in binary on the card which you can convert to a CSV format with the same utility that saves and stores the data.
SdFat has an example called LowLatencyLogger. It predefines the file at like 128MB, then erases the data portion of the file so writes can be done without pre-erase. Writes are then just direct writes to the card's sectors, without messing with the FAT table or the directiory entries. It also uses multiple buffers so you can maintain high logging rates in the face of occasional very slow write times to the card. Then when logging is finished, it redefines the file down to what was actually saved so everything is right with the file system. But if power is lost, the data is still contained in the file. It seems to be a very slick way to handle logging.
Flushing and saving more frequently than 512 byte introduces delays and complications, and generally isn't a good idea. You can't write less than 512 bytes at a time to the card. Just leave the file open until you're done, and accept the fact that a power failure will result in the loss of up to 512 bytes of data.