arduinoJson help

[mesihas]

the JSON data is a webSocket message ( uint8_t * payload )

Then deserialize

DeserializationError err = deserializeJson(doc, payload);

then fetch values.

String updateType = doc["update-type"];

then if I want to use doc later on it won't be available anymore because out of scope? ( not anymore inside function where I did deserialilise ).

I can't figure out how to make the doc (json document) to survive after the end of the function.. because I need the data to avoid making requests with the same data again and again instead of using the data in memory.

Of course at some point I want to make a request and update the data in memory but most of the times I will be working with the data in memory if it were possible.

I am using ESP32 so I think that I could afford memory duplication by creating a copy of the json data or something like that but I don't really know how to do it.

M.

[J-M-L]

do you need the full JSON doc or just the "update-type" information?

there is nothing specific to JSON, if you have a buffer being reused multiple times and you want to collect information along the way, you need another storage repository.

for example if update-type is a number, you could have an array of int (long, ...) and every time you deserialise you store the value in the next available entry in your array. (storing numbers is less memory intensive than storing a collection of Strings)

[david_2018]

If you declare the variable inside a function then it will cease to exist at the end of the function. Declare the variable as a global variable if you need it to persist throughout the entire code, then you can assign a value to it inside the function and use it elsewhere.

[mesihas]

J-M-L:
do you need the full JSON doc or just the “update-type” information?

there is nothing specific to JSON, if you have a buffer being reused multiple times and you want to collect information along the way, you need another storage repository.

for example if update-type is a number, you could have an array of int (long, …) and every time you deserialise you store the value in the next available entry in your array. (storing numbers is less memory intensive than storing a collection of Strings)

I think I like the idea of another repository because I need only part of the original message could be 30% or less, the thing here is that I don’t know how many “scenes” there will be and each “scene” can have multiple sources and the sources have multiple values (fixed qty of fields) and I need to create a data structure that reflects the original message, this could be hard to implement.

The other approach would be to keep the uint8_t * payload alive by copying the whole thing into a new independent array. It does not seem too smart but if the original message and the Json Doc disappear from memory it will be ok to hold a copy?

[J-M-L]

You can dynamically create a a side storage Of json documents, just based from the fields you need, that would save space.

If memory is not an issue then you can keep a set of json documents. Just don’t reuse them to parse your input next time around,

[mesihas]

J-M-L:
You can dynamically create a a side storage Of json documents, just based from the fields you need, that would save space.

If memory is not an issue then you can keep a set of json documents. Just don’t reuse them to parse your input next time around,

Is this something that I can do with the same library? How can I keep these Json doc alive which is the original problem

[J-M-L]

Yes - the library is « just » offering some new types (Classes) for your variables. So it’s like with any other variable.

Just declare an array of pointers to json based storage instances and use new to allocate an entry when needed. new will create a new instance and its lifetime is managed by you. Don’t free it and it will be around “forever”.

You can also use global variables, then they stick around for the lifetime of the program; pre-allocate a max number of those, enough to meet your needs whilst staying within memory constraints - At the end of the day there is a memory limit so be careful

[J-M-L]

So you get a JSON that includes multiple "scene". A scene can have multiple "sources", each having a fixed qty of fields

In your picture above you did not highlight the same fields. Do you need to record all of them

or just locked, muted, name and render that you highlighted (but not consistently)?

[mesihas]

It is exactly as you described it. A scene can have none or multiple sources. I just need (consistently) locked, muted, name and render from sources.

[J-M-L]

Could you share a couple of such json?

[mesihas]

In this example there are 3 scenes. two of them have 2 sources each and last scene has 1 source.

{
  "current-scene": "Welcome",
  "message-id": "11",
  "scenes": [
    {
      "name": "Parte A",
      "sources": [
        {
          "alignment": 5,
          "cx": 0,
          "cy": 0,
          "id": 3,
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true,
          "source_cx": 0,
          "source_cy": 0,
          "type": "coreaudio_input_capture",
          "volume": 0.9885531067848206,
          "x": 0,
          "y": 0
        },
        {
          "alignment": 5,
          "cx": 840,
          "cy": 440,
          "id": 1,
          "locked": false,
          "muted": false,
          "name": "Hello Text",
          "render": true,
          "source_cx": 840,
          "source_cy": 440,
          "type": "text_ft2_source_v2",
          "volume": 1,
          "x": 0,
          "y": 0
        }
      ]
    },
    {
      "name": "LOG CAM",
      "sources": [
        {
          "alignment": 5,
          "cx": 0,
          "cy": 0,
          "id": 2,
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true,
          "source_cx": 0,
          "source_cy": 0,
          "type": "coreaudio_input_capture",
          "volume": 0.9885531067848206,
          "x": 0,
          "y": 0
        },
        {
          "alignment": 5,
          "cx": 1280,
          "cy": 720,
          "id": 1,
          "locked": false,
          "muted": false,
          "name": "Son Cam",
          "render": true,
          "source_cx": 1280,
          "source_cy": 720,
          "type": "av_capture_input",
          "volume": 1,
          "x": 0,
          "y": 0
        }
      ]
    },
    {
      "name": "Parte B",
      "sources": [
        {
          "alignment": 5,
          "cx": 0,
          "cy": 0,
          "id": 2,
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true,
          "source_cx": 0,
          "source_cy": 0,
          "type": "coreaudio_input_capture",
          "volume": 0.9885531067848206,
          "x": 0,
          "y": 0
        }
      ]
    }
  ],
  "status": "ok"
}

[J-M-L]

OK this seems a good job for a filter.

Try this code:

• start a new sketch, save it as test.ino

in test.ino:

#include <ArduinoJson.h>

#include "data.h"

// The filter: it contains "true" for each value we want to keep
StaticJsonDocument<200> filter;
StaticJsonDocument<1000> doc;

void setup() {
  Serial.begin(115200);
  filter["current-scene"] = true;
  filter["scenes"][0]["name"] = true;
  filter["scenes"][0]["sources"][0]["locked"] = true;
  filter["scenes"][0]["sources"][0]["muted"] = true;
  filter["scenes"][0]["sources"][0]["name"] = true;
  filter["scenes"][0]["sources"][0]["render"] = true;

  deserializeJson(doc, (const __FlashStringHelper*) sampleJSON, DeserializationOption::Filter(filter));
  serializeJsonPretty(doc, Serial);
}

void loop() {}

create a second tab in your sketch, call it data.h

in data.h

// C++ raw string literals cf http://en.cppreference.com/w/cpp/language/string_literal
// USE PROGMEM with Program Space Utilities http://www.nongnu.org/avr-libc/user-manual/group__avr__pgmspace.html

const char sampleJSON[] PROGMEM = R"--8<--8<--({
  "current-scene": "Welcome",
  "message-id": "11",
  "scenes": [
    {
      "name": "Parte A",
      "sources": [
        {
          "alignment": 5,
          "cx": 0,
          "cy": 0,
          "id": 3,
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true,
          "source_cx": 0,
          "source_cy": 0,
          "type": "coreaudio_input_capture",
          "volume": 0.9885531067848206,
          "x": 0,
          "y": 0
        },
        {
          "alignment": 5,
          "cx": 840,
          "cy": 440,
          "id": 1,
          "locked": false,
          "muted": false,
          "name": "Hello Text",
          "render": true,
          "source_cx": 840,
          "source_cy": 440,
          "type": "text_ft2_source_v2",
          "volume": 1,
          "x": 0,
          "y": 0
        }
      ]
    },
    {
      "name": "LOG CAM",
      "sources": [
        {
          "alignment": 5,
          "cx": 0,
          "cy": 0,
          "id": 2,
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true,
          "source_cx": 0,
          "source_cy": 0,
          "type": "coreaudio_input_capture",
          "volume": 0.9885531067848206,
          "x": 0,
          "y": 0
        },
        {
          "alignment": 5,
          "cx": 1280,
          "cy": 720,
          "id": 1,
          "locked": false,
          "muted": false,
          "name": "Son Cam",
          "render": true,
          "source_cx": 1280,
          "source_cy": 720,
          "type": "av_capture_input",
          "volume": 1,
          "x": 0,
          "y": 0
        }
      ]
    },
    {
      "name": "Parte B",
      "sources": [
        {
          "alignment": 5,
          "cx": 0,
          "cy": 0,
          "id": 2,
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true,
          "source_cx": 0,
          "source_cy": 0,
          "type": "coreaudio_input_capture",
          "volume": 0.9885531067848206,
          "x": 0,
          "y": 0
        }
      ]
    }
  ],
  "status": "ok"
})--8<--8<--";

compile and run on your Arduino. Serial Monitor (@ 115200 bauds) will show

[color=purple]
{
  "current-scene": "Welcome",
  "scenes": [
    {
      "name": "Parte A",
      "sources": [
        {
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true
        },
        {
          "locked": false,
          "muted": false,
          "name": "Hello Text",
          "render": true
        }
      ]
    },
    {
      "name": "LOG CAM",
      "sources": [
        {
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true
        },
        {
          "locked": false,
          "muted": false,
          "name": "Son Cam",
          "render": true
        }
      ]
    },
    {
      "name": "Parte B",
      "sources": [
        {
          "locked": false,
          "muted": false,
          "name": "AuInCapStreaming",
          "render": true
        }
      ]
    }
  ]
}
[/color]

which is basically this structure:

==> you extracted from your JSON string only the fields of interest and now have that in a json document.

if you receive multiple Scenes and you want to build a merge of those, there is an example in the ArduinoJSON documentation.

does it help?

[sevenoutpinball]

This website is excellent Assistant | ArduinoJson 6 and the author has helped me at least once. JSON is necessary for a lot of API stuff. Luckily once you get it working it's pretty good and not that hard actually.

[mesihas]

J-M-L:
OK this seems a good job for a filter.

does it help?

Yes it does help a lot to reduce json Doc size.
In you rexample the payload lives forever, but in my world as soon as I receive a message the payload will be replaced with a new message and the json Doc data will die.
Is it possible to populate a new array like in you example but in running time?

[J-M-L]

As hinted in my answer, you could have a working json donc to filter and extract the required elements from the incoming JSON message and a global JSON (much larger in size) in which you merge the working json before going to receive the next message. Over. Time the global json accumulates all the temporary ones.

[mesihas]

J-M-L:
As hinted in my answer, you could have a working json donc to filter and extract the required elements from the incoming JSON message and a global JSON (much larger in size) in which you merge the working json before going to receive the next message. Over. Time the global json accumulates all the temporary ones.

Thanks, I am implementing this now.