How to replace a variable lenght record on a SD card

Hello,

I have "myFile.txt" on a Sd card which holds just one value "Good evening Mark".
After reading this record I need to rewrite/replace it with "Welcome John".
Is there a better way to do this rather than removing the .txt file and re creating it ?

If there is a known maximum number of characters to be stored then you could save the data in fixed length records of that size thus allowing them to be overwritten without re-writing the whole file

This wastes space, but unless the number of records is huge it is not likely to be a problem

you can just overwrite if the new content has the same size or is longer than the previous content.

the SdFat library supports the truncate() method but the standard builtin SD library does not and you would have to delete and recreate

with the SD library you could explore what you can do with the O_TRUNC mode

  • O_TRUNC - If the file exists and is a regular file, and the file is successfully opened and is not read only, its length shall be truncated to 0.

as a way to just replace the content that was there (if that's the intent)

Another approach is to have some data formatting rules, for example you could say that the meaningful data is on the first line - up to the CR/LF mark for example. This way when you overwrite, you insert the end marker and whatever is still in the file after the mark is just to be ignored. (but if security is important you don't want that as previous data might still be left behind).

I know what you meant, but it is not what you wrote :grinning:

lost in translation again ? :slight_smile:

what I meant is : open the file for writing, seek to position 0, write the new text. close the file

what did I write?

@ba47 wants to replace a single record in a file for variable length records so cannot just write a longer record in the place of a smaller one, even assuming that he/she knows where to seek() to

I assumed that you meant
you can just overwrite if the new content has the same size or is shorter than the previous content.

Ah, no I really meant longer because @ba47 said

so my understanding was that there was no (meaningful) data after that text in the file.

Thank you both.
I DO NOT have fixed length data and don't know how to use the suggested CR/LF. Do you think the easy way (which I know how to do) i.e. removing and recreating the .txt file will create problems ?

I would suggest to move to the SDFat library. That library is overall better than the builtin SD library and then just truncate after writing the new content.

Writing CR/LF is just done by using println() to write your text. That will add the two end markers. Reading the content then could be done by writing a readLine function or just read whatever is in the file and trim after the CR/LF

There is no real harm in deleting the file and recreating one, besides the wear and tear for your SD card - but that happens to some extent as well when you replace the content anyway. if you don't do this 10 times per second every second of the day, that should be no problem

Thank you.

@J-M-L

Sorry, I missed that

If the file only contains a single record then I see little point in using an SD card in the first place. You might just as well use EEPROM if you want the sketch to have access to persistent data.

Doesn't EEPROM wear out after a while ?

Each EEPROM location has a nominal life of 100,000 write/erase cycles. How often will you be writing a new record ?

You can also extend the life of the EEPROM by not always writing to the same locations

Thanks, 100,000 is good enough. Your suggestion saves extra modules and wiring.

If you use EEPROM.put() to save the data then only EEPROM cells where the data is changing will actually be written, which may also help prolong the life of the EEPROM with no extra effort

You beated me to it. I didn't want to bother you asking how to write/read EEPROM. Never have used it, so I'll study. Will bother you in case I get stuck.
Thanks again.

There are examples in the IDE

Basically you put the string (not a String) of characters to be saved in a zero terminated array of chars then use the EEPROM.put() function to save it. To read it back you declare an array of chars large enough to hold the string then use the EEPROM.get() function to populate the string

dealing with an SD card, you would know if the file is missing before reading the data.
In case of EEPROM, you just have bytes available but don't know if it's relevant data or garbage (the first time you try to read, the EEPROM has not been initialized and thus the get() will return garbage.

A way to deal with this is to add a magic keyword (say an unsigned long) at a known address in EEPROM. When you want to read the string, you first perform a get() at the magic keyword address and if you find the expected data then you can assume the string is correctly stored at the next address. if you don't find the keyword, then you need to have some error handling mechanism, possibly providing a default value (and writing it to the EEPROM with the keyword for next time).

Ok, thanks.