Parse Json download size

B1Gtone · January 23, 2022, 1:52pm

I have the following as part of a sketch that gets API data from Open Weather.
This works fine if I download the 'daily' API string that is, on average, 4156 characters long.
If I try to download more data with the 'hourly' API then the download fails Serial.print("parseObject() failed);

My question is, how long a string can I download? Is there a limit set by this code or within the Json library?
I have tried changing the #define JSON_BUFF_DIMENSION 2500 value but still no joy.

#include <ArduinoJson.h>
#define JSON_BUFF_DIMENSION 2500
  text.reserve(JSON_BUFF_DIMENSION);


void parseJson(const char * jsonString) {
  DynamicJsonBuffer jsonBuffer;
  JsonObject& root = jsonBuffer.parseObject(jsonString);
  if (!root.success()) {
    Serial.println("parseObject() failed");
    return;
  }
void.loop(){
//display stuff
}

J-M-L · January 23, 2022, 2:16pm

Don't post snippets (Snippets R Us!)

Are you using ArduinoJson 5 ?
the DynamicJsonBuffer is allocated on the heap, so can grow big (as long as you have memory).

B1Gtone · January 23, 2022, 2:35pm

Yes I am using ArduinoJson 5

'On the heap'?

J-M-L · January 23, 2022, 2:46pm

as opposed to on the stack.

johnwasser · January 23, 2022, 3:05pm

Might be time to update to ArduinoJson 6. The "Assistant" can write most of the code for you:

B1Gtone · January 23, 2022, 5:56pm

I don't have a problem extracting the data I require which is all the assistant helps with.
My problem is parsing a large amount of data

johnwasser · January 23, 2022, 6:03pm

Can you show an example of the large JSON document you are having trouble parsing?

B1Gtone · January 23, 2022, 6:09pm

B1Gtone · January 23, 2022, 6:11pm

This one works

johnwasser · January 23, 2022, 6:18pm

The one that works has a minimum size of 5258 (Recommended: 6144). The one that doesn't work has a minimum size of 21674 (Recommended: 24576).

Could it be that you are running out of memory on the larger one?

B1Gtone · January 23, 2022, 6:26pm

Is there a way I could test this?

johnwasser · January 23, 2022, 6:36pm

If you use ArduinoJson 6 the deserializeJson() function will return a DeserializationError object that will tell you WHY the deserialization failed.

DynamicJsonDocument doc(6144);

DeserializationError error = deserializeJson(doc, input);

if (error) {
  Serial.print(F("deserializeJson() failed: "));
  Serial.println(error.f_str());
  return;
}

B1Gtone · January 23, 2022, 7:06pm

I did try this but it would not compile. It said something along the lines of deserializeJson() is not part of ArduinoJson 6

johnwasser · January 23, 2022, 7:09pm

I would guess something along the lines of a library installation error.

B1Gtone · January 23, 2022, 7:17pm

This is what I get when I try to compile:
no matching function for call to 'deserializeJson(ArduinoJson6191_F1::DynamicJsonDocument&)'

johnwasser · January 23, 2022, 7:28pm

It looks like nothing is being passed to the parser. You need to pass it the JSON document in some way. You can pass a String, character pointer, const character pointer, or a Stream (serial port, network client, or file) to read the data from.

B1Gtone · January 24, 2022, 4:21pm

I have updated ArduinoJson.h to v6 and downloaded the example parse sketch as shown below.
Each time I press reset it Serial.print's the required information.

I have added the first section of the weather API as a test (Line 14) but if I un-comment this line the first two words, 'Lat' and 'Lon' are highlighted in orange and I get the error 'inconsistent user-defined literal suffixes 'lat' and 'lon' in string literal'

[code]

#include <ArduinoJson.h>

void setup() {

  Serial.begin(9600);
  while (!Serial) continue;

  DynamicJsonDocument doc(200);

  char json[] =
 "{\"sensor\":\"gps\",\"time\":1351824120,\"data\":[48.756080,2.302038]}";

//"{"lat":52.4976,"lon":-2.1689,"timezone":"Europe/London","timezone_offset":0,"current":}";

  DeserializationError error = deserializeJson(doc, json);

  if (error) {
    Serial.print(F("deserializeJson() failed: "));
    Serial.println(error.f_str());
    return;
  }


  const char* sensor = doc["sensor"];
  long time = doc["time"];
  double latitude = doc["data"][0];
  double longitude = doc["data"][1];

  Serial.println(sensor);
  Serial.println(time);
  Serial.println(latitude, 6);
  Serial.println(longitude, 6);
}

void loop() {

}

[/code]

J-M-L · January 24, 2022, 4:33pm

if you inject in the char array the json file you shared earlier, as it contains double quotes, you mess up with the C++ string definition. You either need to escape those double quotes (write them as ") or go for a string literal

B1Gtone · January 24, 2022, 4:58pm

As it's a downloaded file I cannot edit in any way.

How would I do this?

J-M-L · January 24, 2022, 5:02pm

you don't need to do that when you download, but if you want to inject that into your code for testing (as static text) then you need to.

an example with a string literal:

create a project with those 2 files.

test.ino

#include <ArduinoJson.h>
#include "json.h"

StaticJsonDocument<200> filter;
StaticJsonDocument<300> doc;

void setup() {

  Serial.begin(115200);

  filter["lat"] = true;
  filter["lon"] = true;
  filter["daily"][0]["sunrise"] = true;
  filter["daily"][0]["sunset"] = true;
  deserializeJson(doc, (const __FlashStringHelper*) sampleJSON, DeserializationOption::Filter(filter));
  serializeJsonPretty(doc, Serial);
}

void loop() {}

I store the JSON data separately in this file

json.h

// C++ raw string literals cf http://en.cppreference.com/w/cpp/language/string_literal
// USE PROGMEM with Program Space Utilities http://www.nongnu.org/avr-libc/user-manual/group__avr__pgmspace.html

const char sampleJSON[] PROGMEM = R"--8<--8<--({
  "lat":52.4976,
  "lon":-2.1689,
  "timezone":"Europe/London",
  "timezone_offset":0,
  "current":{
    "dt":1642961465,
    "sunrise":1642925007,
    "sunset":1642955830,
    "temp":2.99,
    "feels_like":0.24,
    "pressure":1035,
    "humidity":86,
    "dew_point":0.88,
    "uvi":0,
    "clouds":96,
    "visibility":10000,
    "wind_speed":2.82,
    "wind_deg":193,
    "wind_gust":4.69,
    "weather":[
      {
        "id":804,
        "main":"Clouds",
        "description":"overcast clouds",
        "icon":"04n"
      }
    ]
  },
  "daily":[
    {
      "dt":1642939200,
      "sunrise":1642925007,
      "sunset":1642955830,
      "moonrise":1642979040,
      "moonset":1642933980,
      "moon_phase":0.68,
      "temp":{
        "day":4.15,
        "min":0.52,
        "max":4.15,
        "night":0.52,
        "eve":2.99,
        "morn":0.88
      },
      "feels_like":{
        "day":1.06,
        "night":-2.58,
        "eve":0.24,
        "morn":-1.1
      },
      "pressure":1035,
      "humidity":84,
      "dew_point":1.55,
      "wind_speed":3.8,
      "wind_deg":197,
      "wind_gust":4.86,
      "weather":[
        {
          "id":803,
          "main":"Clouds",
          "description":"broken clouds",
          "icon":"04d"
        }
      ],
      "clouds":68,
      "pop":0,
      "uvi":0.72
    },
    {
      "dt":1643025600,
      "sunrise":1643011331,
      "sunset":1643042338,
      "moonrise":0,
      "moonset":1643021160,
      "moon_phase":0.71,
      "temp":{
        "day":4.49,
        "min":-0.04,
        "max":5.47,
        "night":0.65,
        "eve":1.6,
        "morn":0.13
      },
      "feels_like":{
        "day":4.49,
        "night":0.65,
        "eve":1.6,
        "morn":-1.39
      },
      "pressure":1034,
      "humidity":78,
      "dew_point":0.8,
      "wind_speed":2.74,
      "wind_deg":187,
      "wind_gust":4.95,
      "weather":[
        {
          "id":800,
          "main":"Clear",
          "description":"clear sky",
          "icon":"01d"
        }
      ],
      "clouds":0,
      "pop":0,
      "uvi":0.71
    },
    {
      "dt":1643112000,
      "sunrise":1643097652,
      "sunset":1643128846,
      "moonrise":1643070180,
      "moonset":1643108460,
      "moon_phase":0.75,
      "temp":{
        "day":4.72,
        "min":-0.24,
        "max":5.81,
        "night":0.93,
        "eve":1.38,
        "morn":-0.08
      },
      "feels_like":{
        "day":4.72,
        "night":-1.53,
        "eve":-0.73,
        "morn":-0.08
      },
      "pressure":1034,
      "humidity":74,
      "dew_point":0.33,
      "wind_speed":2.13,
      "wind_deg":294,
      "wind_gust":2.86,
      "weather":[
        {
          "id":800,
          "main":"Clear",
          "description":"clear sky",
          "icon":"01d"
        }
      ],
      "clouds":0,
      "pop":0,
      "uvi":0.71
    },
    {
      "dt":1643198400,
      "sunrise":1643183971,
      "sunset":1643215356,
      "moonrise":1643161620,
      "moonset":1643195940,
      "moon_phase":0.78,
      "temp":{
        "day":5.85,
        "min":0.33,
        "max":6.51,
        "night":2.46,
        "eve":3,
        "morn":0.36
      },
      "feels_like":{
        "day":4.7,
        "night":-1.19,
        "eve":0.08,
        "morn":-1.18
      },
      "pressure":1034,
      "humidity":72,
      "dew_point":1.07,
      "wind_speed":3.89,
      "wind_deg":222,
      "wind_gust":9.3,
      "weather":[
        {
          "id":803,
          "main":"Clouds",
          "description":"broken clouds",
          "icon":"04d"
        }
      ],
      "clouds":63,
      "pop":0,
      "uvi":0.76
    },
    {
      "dt":1643284800,
      "sunrise":1643270287,
      "sunset":1643301866,
      "moonrise":1643253240,
      "moonset":1643283720,
      "moon_phase":0.82,
      "temp":{
        "day":8.65,
        "min":2.69,
        "max":8.65,
        "night":3.06,
        "eve":5.28,
        "morn":5.92
      },
      "feels_like":{
        "day":5.66,
        "night":-0.12,
        "eve":1.93,
        "morn":2.12
      },
      "pressure":1027,
      "humidity":91,
      "dew_point":7.06,
      "wind_speed":5.85,
      "wind_deg":249,
      "wind_gust":14.05,
      "weather":[
        {
          "id":804,
          "main":"Clouds",
          "description":"overcast clouds",
          "icon":"04d"
        }
      ],
      "clouds":99,
      "pop":0.19,
      "uvi":0.31
    },
    {
      "dt":1643371200,
      "sunrise":1643356601,
      "sunset":1643388377,
      "moonrise":1643344920,
      "moonset":1643372100,
      "moon_phase":0.86,
      "temp":{
        "day":7.03,
        "min":2.61,
        "max":7.89,
        "night":3.04,
        "eve":4.12,
        "morn":3.03
      },
      "feels_like":{
        "day":5.01,
        "night":-0.09,
        "eve":1.87,
        "morn":-0.21
      },
      "pressure":1038,
      "humidity":72,
      "dew_point":2.23,
      "wind_speed":5.25,
      "wind_deg":313,
      "wind_gust":13.18,
      "weather":[
        {
          "id":803,
          "main":"Clouds",
          "description":"broken clouds",
          "icon":"04d"
        }
      ],
      "clouds":66,
      "pop":0,
      "uvi":0.85
    },
    {
      "dt":1643457600,
      "sunrise":1643442913,
      "sunset":1643474889,
      "moonrise":1643436240,
      "moonset":1643461440,
      "moon_phase":0.9,
      "temp":{
        "day":12.31,
        "min":3.74,
        "max":12.31,
        "night":6.56,
        "eve":9.66,
        "morn":8.09
      },
      "feels_like":{
        "day":11.42,
        "night":3.64,
        "eve":7.54,
        "morn":5.08
      },
      "pressure":1032,
      "humidity":70,
      "dew_point":6.83,
      "wind_speed":6.74,
      "wind_deg":266,
      "wind_gust":13.52,
      "weather":[
        {
          "id":804,
          "main":"Clouds",
          "description":"overcast clouds",
          "icon":"04d"
        }
      ],
      "clouds":87,
      "pop":0,
      "uvi":1
    },
    {
      "dt":1643544000,
      "sunrise":1643529223,
      "sunset":1643561401,
      "moonrise":1643526780,
      "moonset":1643551800,
      "moon_phase":0.93,
      "temp":{
        "day":7.3,
        "min":4.38,
        "max":7.3,
        "night":4.38,
        "eve":5.8,
        "morn":6.38
      },
      "feels_like":{
        "day":4.14,
        "night":0.69,
        "eve":2.31,
        "morn":3.02
      },
      "pressure":1036,
      "humidity":65,
      "dew_point":1.05,
      "wind_speed":5.68,
      "wind_deg":312,
      "wind_gust":12.02,
      "weather":[
        {
          "id":804,
          "main":"Clouds",
          "description":"overcast clouds",
          "icon":"04d"
        }
      ],
      "clouds":100,
      "pop":0,
      "uvi":1
    }
  ]
})--8<--8<--";

if all goes well, Serial Monitor (@ 115200 bauds) will show

{
  "lat": 52.4976,
  "lon": -2.1689,
  "daily": [
    {
      "sunrise": 1642925007,
      "sunset": 1642955830
    },
    {
      "sunrise": 1643011331,
      "sunset": 1643042338
    },
    {
      "sunrise": 1643097652,
      "sunset": 1643128846
    },
    {
      "sunrise": 1643183971,
      "sunset": 1643215356
    },
    {
      "sunrise": 1643270287,
      "sunset": 1643301866
    },
    {
      "sunrise": 1643356601,
      "sunset": 1643388377
    }
  ]
}

==> that example shows how you can put filters to work for you when you have tons of data

Topic		Replies	Views
Decoding json files Programming	11	446	January 16, 2024
Http post with a long payload & read json response Programming	33	740	December 9, 2025
Json Parsing Issue Programming	19	156	November 22, 2025
ArduinoJson v7 deserialization problem Programming	13	837	October 4, 2024
How to minimize the size of a json file Programming	28	310	February 14, 2025

Parse Json download size

Related topics