I am having trouble understanding how to take data logged to the SD card (say, every 5 minutes) and then periodical send the new data to a google spreadsheet (every hour).
The SD card example shows appending a file, I was thinking that I would have to keep a variable of the first line number that was appended after the last write to google Spreadsheet and go through a loop that reads from SD and writes to google until the data aren't available in the SD file.
Alternatively, I thought to create a new file on the card after the upload, so there would be 24 files per day in my example above.
I am looking for something pretty easy. I even thought to just upload the data to dropbox but I still run into the same problem. Imagine uploading a file that is growing bigger and bigger to overwrite the old one each time. That seems very inefficient.
Full disclosure: Any help with or without code is So very appreciated. This is for a research project I am doing as a (non-Federal) postdoc at the EPA. I will reference any help I receive from the community here. - Thank you.
Do you have some code to show? What problems are you facing?
If you have to keep track of the lines you've already sent and don't want to read the whole file every hour, I would use a database. sqlite3-cli is the package, to keep things simple.
Thank you Federico. When I found out that the power consumption was very high, I decided to do on-device storage on an RFDuino for the time being. Still, I will come back to this and deploy one just to see how long the logger lasts.
This is operating at a higher frequency than when I deploy. The reading will be taken and the file appended every 5 minutes.
Now, using SQL may be a better idea. Can you describe what you mean? I think would be more efficient if I could write to a db onboard and then use something (ODBC?) to sync the newly added on-device records to the online DB. May be complicated though. Any help is appreciated.
A negative to using a DB onboard is that plain text is often easier to work with for field researchers and can be partially retrieved if the file gets corrupted.