can we sort a data file (.txt / .csv) from an sd card in Arduino ?

I made a data logger with Arduino Due and SD card Shield. I am using an RTC module along with two potentiometers to generate two analog inputs along with a time stamp. I am data logging the two values with the time stamp in the SD card at a small interval. Can I use the text file or the .CSV file to extract the data and send the value periodically to IOT cloud for every one minute? Let us say I am using an interval of one second. Can i get the data separately like 60 individual values in an array so that i can update it. The data should contain the first 60 values then the next 60 values and so on. I am planning to device this like a standalone system. Is there any way to extract the data from the text file in the Arduino Board itself without using any additional softwares ? Please guide me regarding the same.
Thank you in advance

Regards,
Kumaresh.

Mate you just open them as text files.

The way you can process the text they contain is limited only by your programming knowledge/ability.

I appreciate your time and effort for answering my question. I tried to open the text file and all I am able to do is open the file completely or nothing at all. I just do not know how to make a data sorter/ data buffer to load the first 60 values and so on. I am new to programming and I am trying to import all the knowledge i can get but i just do not know how to implement it. The perfect example for what i am looking at is the Biometric attendance system. it contains an SD card to data log all the values but we can choose to get the required specific data of the lot like 01/Jan /2018 to 31/Jan /2018. I kind of feel dumb in advanced programming :slightly_frowning_face: . Please guide me to make a data buffer.
Thanks in advance,
Regards,
Kumaresh.

You open the file.
You sequentially read the records one at a time until you get to the first one you need and you dave that in an array or some other memory.
You continue to sequentially read the records one at a time and keep saving them to your array until you get to the last one.

If your records are all the same size (characters or bytes), then you jump to a specific location in the file using seek(). This would be the record number (if you know it) times the size of the records.

There are other techniques like creating an index file that has the index for the record, 'sort' field (like date), and the location in the file (as a byte offset) to allow you jump directly to records, etc. This may be too advanced for you right now.

marco_c:
You open the file.
You sequentially read the records one at a time until you get to the first one you need and you dave that in an array or some other memory.
You continue to sequentially read the records one at a time and keep saving them to your array until you get to the last one.

If your records are all the same size (characters or bytes), then you jump to a specific location in the file using seek(). This would be the record number (if you know it) times the size of the records.

There are other techniques like creating an index file that has the index for the record, 'sort' field (like date), and the location in the file (as a byte offset) to allow you jump directly to records, etc. This may be too advanced for you right now.

Keeping in find how big the files are likely to be. If they are of the order of kB in size then you will run out of Arduino memory.

It might be better to make use of the seek(...) function which will allow you to skip over all the irrelevant records in the file to the one in the file you want..

This is assuming you are storing values of a fixed and known size (in bytes).

Agree with marco_c about fixed width.

Alternatively, store in file for record keeping but do not read from the file.Keep the last 60 entries in memory and send when needed.