Read the last value from SD file

Other Hardware Storage
1

Greeting,

Would you please help me with a sketch to seek/read the last value written on a file in an SD card? I am using a project to calculate kWh, and this value must be incremented, but when the Arduino/ESP restart powered off, the kWh reset to zero. Is it possible to get the last stored value and stored value and start incrementing from it? Thanks for your support.
The log file looks like
Date, Time , kWh
26/12/2022,03:11:42,0.14
26/12/2022,03:11:43,0.23
26/12/2022,03:11:44,0.28
26/12/2022,03:11:45,0.46
26/12/2022,03:11:47,0.67
26/12/2022,03:11:49,0.81
26/12/2022,03:11:51,0.97
26/12/2022,03:11:52,1.09
26/12/2022,03:11:54,1.15
26/12/2022,03:11:56,1.21
.
.
.
.
27/12/2022,03:48:23,44.52
27/12/2022,03:48:25,46.65

2

Yes. Starting from the beginning, read a line into a buffer. Keep on reading and placing into (not adding to) the buffer. Once you no longer can read from file, the last content of the last line is in the buffer.

Your problem has absolutely nothing to do with Installation and Troubleshooting and hence has been moved to a more suitable location on the forum.

3
4

It looks like each line in the file has the same length (i.e. 24 chars), so in order to reach the last one you don't need to rewind it all line by line.
Just make a seek() to the file address (filelength - 24) and read the line in the variable.

5

Thank you so much for your reply. The value of the kWh keeps incrementing, so after a few hours/days, the length would be increased, too.

6

Thank you so much for your response. I needed to read only the last value, not the line; the issue is the last value length changed over time
26/12/2022,03:11:42,0.14
27/12/2022,03:48:25,46.65
so the length of the line before the kWh value starts is 20.

7

You need to find the last comma in the file and read the value after it

8

I do appreciate your support and would be more appreciative if you could post the sketch. I am not professional and learning more every day. Thank you so much.

9

I think you need to do some "file handling"..

For example you could find/count all the new line (\n character).
Once you know the number of lines, you can "set the cursor" to the last one.

Another possibile way could be reading all the content of file up to the end and then move back until you find the comma char in the stream.

For example

  uint16_t commaPos;
  while(SD_File.avalaible()){
    char cr = SD_File.read();
    if(cr == ',')
        commaPos = SD_File.position();
   }
  SD_File.seek(commaPos);
  // Read your value
  float value = SD_File.parseFloat();
  SD_File.close();
1 Like
10

Do you know when you use FileSystem (e.g. FatFS) there is also an option to enable all the other file system functions, e.g. seek or fseek?
You can set the read/write pointer from/to file via this function.
If you know the size of the last string (or word) written, you can set the pointer for next read also to the tail of the file, e.g. count the bytes written in total to know where the end is.
I think, you can do also "relative" addressing: instead of setting pointer to a (byte) address, you can potentially also say: "set it to end of file MINIUS x bytes".
If not, I think seek / fseek can tell you also the current file size (where is end of file). So, you can go from the end of file and set the pointer a bit before end, at the right place when you know the string length of the tail.

The File System should support to move pointer on an open file. Potentially, there is macro for FatFS config to set, in order to enable the full-blown functions generated.
It should work like POSIX standard (like a UNIX filesystem). But I guess there is also a macro if you want to have POSIX style or a proprietary style.

Check, if you have seek() or fseek() available. If so - it will do the job.

1 Like
11

Thank you so much @cotestatnt ; that sketch worked ideally and as I wished. I do appreciate your kind support.

12

COOL. Well done!

13

A very inefficient way especially for long files.

You don't need to search whole file, if you want to find the last comma. Looking in the last 100 bytes will be enough:

uint32_t commaPos = SD_File.size();
if (commaPos > 120)  SD_File.seek(commaPos-100);
while(SD_File.avalaible()){
  char cr = SD_File.read();
  if(cr == ',')
      commaPos = SD_File.position();
}
SD_File.seek(commaPos);
  // Read your value
float value = SD_File.parseFloat();
SD_File.close();

And note that that the return value from Position() call should be unsigned long.

1 Like
14

If the file consists of fixed length records, then simply use fseek to position the file pointer to where you want to start reading from.

15

This were discussed in post#4

16

If the OP were to pad their readings with leading zeros then it would be a much cleaner solution.

17

You are right.
If the schema of data will be always more or less the same, you can seek() the file even with less bytes.

1 Like
18

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.