[JSON] Best pratice to savec measure in JSON format into a SD card

Other Hardware Storage
1

Hi,

I am actually working to save several measure into a SD card. I want to save it in JSON formar with Arduinojson library.

I am not very sure about how to save in a correct JSON format, as each measure loop must have a record. For example, the board measure each 10mn the temperature, the pressure, the humidity and tha altitude. When the measure is done, it must save the measures into the SD card
{"time":"2024-4-19 23:32:38","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]}
Next time, it does the save and the record are saved in the next line

{"time":"2024-4-19 23:32:38","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]}
{"time":"2024-4-19 23:32:49","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]}

...a dn so on
Is it consider as. a JSON format for you? I am asking this because I expected to have

[{"time":"2024-4-19 23:32:38","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]},
{"time":"2024-4-19 23:32:49","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]}]

with the [] and the , beween set of measure

I wonder, what you suggest to do it correctly?

Cheers

2

You can use this site to verify if your json is right. " https://jsonviewer.stack.hu/

Copy your json and past at text tab.
After, see if correct at Viewer.

For ex: This json is invalid,

":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]}
{"time":"2024-4-19 23:32:49","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]}

but this is right:

[{"time":"2024-4-19 23:32:38","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]},
{"time":"2024-4-19 23:32:49","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]}]
3

Salut @pierrot10

You could have an opening [ that you keep as the first element of the file and a ] at the end to build a JSON array

When you want to add an entry to the file, don’t add it after the last byte, ask for the file size and seek() to that size minus 1
This way you overwrite the closing square bracket and you can write your new entry (don’t forget the coma if it’s not the first entry). Then, before closing the file, write again the ] you had overwritten.

This way the file maintains its structure.

4

Hello @J-M-L
It's very interesting, I am going to investigate about how to. I let you know.

5

While investigating about your suggestion, I found this. It's interresting, isn't?

But I prefere to have the data saved (into the SD) as the following

[
  {"first_name":"Stan","last_name":"Marsh"},
  {"first_name":"Kyle","last_name":"Broflovski"},
  {"first_name":"Eric","last_name":"Cartman"}
]

(with the [] and , )

I am just investigateing how... :slight_smile:

6

I would lean toward that JSON Lines format, also called ndJSON (for newline-delimited), where each line is a complete JSON object. This allows for easy appending -- you don't have to play with the separating comma, and no [ nor ]. This is a common format for logs and "lots of data", for example with AWS CloudWatch and Google BigQuery.

{"time":"2024-4-19 23:32:38","data":[{"temperature":21.21999931},{"pressure":936.6514282},{"altitude":658.1940308},{"altitude":100}]}

Always use two digits for the month and day, and always in YYYY-MM-DD format. In fact, you can avoid ambiguity by using ISO 8601 Zulu time. Why are there two "altitude"? There may not be enough value in nesting the data under "data".

{"time":"2024-04-19T22:32:38Z","temperature":21.21999931,"pressure":936.6514282,"altitude":658.1940308}
{"time":"2024-04-19T22:32:49Z","temperature":21.220001,"pressure":936.6514282,"altitude":658.1940308}

If you use jq, it reads JSON Lines with --slurp or -s

$ xsel --clipboard --output | jq --s
[
  {
    "time": "2024-04-19T22:32:38Z",
    "temperature": 21.21999931,
    "pressure": 936.6514282,
    "altitude": 658.1940308
  },
  {
    "time": "2024-04-19T22:32:49Z",
    "temperature": 21.220001,
    "pressure": 936.6514282,
    "altitude": 658.1940308
  }
]

and outputs with --compact-output or -c and '.[]'

$ xsel --clipboard --output | jq -s | jq -c '.[]'
{"time":"2024-04-19T22:32:38Z","temperature":21.21999931,"pressure":936.6514282,"altitude":658.1940308}
{"time":"2024-04-19T22:32:49Z","temperature":21.220001,"pressure":936.6514282,"altitude":658.1940308}
7

That’s the JSONLine format that @kenb4 also referred to. It depends what you want to do with the log and how you want to parse it in the future.

One advantage is for large files as you don’t need to go read it all to understand the json structure. You parse it as it comes and extract one line at a time.

8

Thanks a lot.
I will go with ndJSON. It's look easier to write data to an sd card. I am going to continue my script (write/read the sd card) later

Always use two digits for the month and day

OK. Thanks for your advise

9

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.