How to makes log rotate in sd card

i want to make a program to delete the first row in the txt file in creating a datalogger project using sdCard and RTC

for example :
1.
2.
3.
4.
5.

then it will be:
2.
3.
4.
5.
6.

then it will be:
3.
4.
5.
6.
7.

and so on repeated with a pattern like this. Can arduino makes the pattern like that ? get someone help me with it, thanks in advance

There is no simple way to do that.

One way to proceed is to copy the data in the original file to a new file, leaving out that first row, then close the new file, delete the original file, and rename the new file to the original name.

A line of new data can then be added to the resulting file.

Do you really only need 5 data points or was that just for example?
You mentioned RTC. Are you writing the time/date with your data?

Maybe so.

  • You want 5 rows.
  • Create a file with 6 blank lines.
  • In the first line, write the number of the actual work line and refresh after each saved line.
  • After writing the last line, rewrite the second line and so on.

Just an idea, not tested.

how to leave the first line ? I managed to make a pattern when 5 data has been entered, then the file will be deleted and will be
create a new file again, but I'm still confused to do what I meant as in the beginning log rotate process.

that is my code
5_data.ino (2.1 KB)

yes 5 data its only example. yes i used rtc and writing the time/date with data

i am confused to make it, can you share for some file for my learning about that ?

Open the file. Open another file for write, call it file 2.
Read one line. Discard it (do nothing).
While the end of file is not reached
{
read a line from file
write it to file 2
}
close file and file 2

Why do you want to do this? @aarg has given you the answer, but as you can see it's painful. Perhaps there's a better way.

Oh yeah, I forgot...

rename file2 to file
delete file 2

A lot of repetitive file write/deletes can cause fragmentation in some file systems. You can sometimes do indexed file access, but moving the lines in place is an awkward process.

There really is no easy way to do what you want, and the more writes you make to the SD card, the greater the chance of a fatal error that could corrupt all the data on the card.

What is the data logging rate? You have gigabytes of storage on the card... it might take years to fill the card, depending on the rate.

Some applications like this, just open a new file for each new day, or month, week etc.

sorry sir, can you apply it directly to my scripth directly? I'm still confused about what that means. thank you in advance

5_data.ino (2.1 KB)

Yes, you can.

I'm logging data per second, the reason I'm doing this log rotate is trying to keep this storage capacity fixed with that capacity not increasing again

I already offered suggestions for that, please respond to them. Reply #12.

Also, "data per second" does not answer my question about the data rate. I would also need to know how much data per second. Obviously.

Also, have you tried any of the file pseudo code that I provided?

Also, how big is the file (how many seconds data are you trying to keep)?

I save the data to the SD card according to the data sheet storage speed
which is about 2mb/second. I save 3 sensor data
namely ds18b20, turbidity and ph sensors.

I'm trying it but still confused

for the size of the file every 1 minute the data is maintained, so there are approximately 60
data that will last

does that answer everything? sorry my english is a bit limited

So, 60 bytes every second? 60x60x24 seconds in a day. Tell me, how many bytes per day?

Also, what is your level of coding experience? Did you ever work with files before?

yes 1 second is 60 bytes. every day in total for 24 hours there are 5,184,000 bytes

i'm a newbie, i've never worked on that file it's just that i've implemented
in the data logger process whose data never stops until the memory is full

You have two choices. You can recreate the file each time (which will eventually crash the card), or you can open the file in indexed mode and make a circular buffer there. It's far too complicated to explain the whole thing here, it would amount to doing the entire job for you. You should contemplate a simpler method like saving a daily log and deleting them later, if you can't manage the programming for a circular buffer file.

However, you could log about 1000 days of 5m of data per day, on an 8Gb card. Are you expecting data to be intact if power is lost? If not, you will have to open and close the file each time you access it.

I think the data size got lost in translation, you never told me how big a single log entry is.