Parsing JSON from wunderground https get request using ArduinoJSON version 6

Projects Programming
1

I have been trying to get the data from wunderground to parse so I can use certain parts of it...
like: "dayOfWeek" and "narrative"

I was able to get the long JSON string returned however I cannot for the life of me figure out how to use the ArduinoJSON library to parse from a returned string...

I visited Assistant | ArduinoJson 6 and put in the returned JSON string from wunderground that gave me this:

Expression
JSON_ARRAY_SIZE(1) + 20JSON_ARRAY_SIZE(6) + 27JSON_ARRAY_SIZE(12) +
JSON_OBJECT_SIZE(21) + JSON_OBJECT_SIZE(27)

ESP32/ESP8266 7888+4268 = 12156

but I dont understand what to do with it..

here is my current code:

/**
   BasicHTTPSClient.ino
    Created on: 20.08.2018
*/
#include <ArduinoJson.h>
#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClientSecureBearSSL.h>


// Current fingerprint for https://api.wunderground.com
const uint8_t fingerprint[20] = {0x5D, 0xC3, 0x85, 0xD6, 0x19, 0x22, 0x99, 0x2B, 0x7B, 0xBB, 0x87, 0x8F, 0xD6, 0x9E, 0xF2, 0xD5, 0x44, 0xDA, 0x32, 0x28};


ESP8266WiFiMulti WiFiMulti;

void setup() {

  Serial.begin(115200);
  // Serial.setDebugOutput(true);

  Serial.println();
  Serial.println();
  Serial.println();

  for (uint8_t t = 4; t > 0; t--) {
    Serial.printf("[SETUP] WAIT %d...\n", t);
    Serial.flush();
    delay(1000);
  }

  WiFi.mode(WIFI_STA);
  WiFiMulti.addAP("----REDACTED----", "----REDACTED----");
}

void loop() {
  // wait for WiFi connection
  if ((WiFiMulti.run() == WL_CONNECTED)) {
    std::unique_ptr<BearSSL::WiFiClientSecure>client(new BearSSL::WiFiClientSecure);
    client->setFingerprint(fingerprint);
    HTTPClient https;
    Serial.print("[HTTPS] begin...\n");
    if (https.begin(*client, "https://api.weather.com/v3/wx/forecast/daily/5day?postalKey=78249:US&units=e&language=en-US&format=json&apiKey=------------REDACTED------------")) {
      Serial.print("[HTTPS] GET...\n");
      int httpCode = https.GET();
      if (httpCode > 0) {
        Serial.printf("[HTTPS] GET... code: %d\n", httpCode);
    if (httpCode == HTTP_CODE_OK || httpCode == HTTP_CODE_MOVED_PERMANENTLY) {
      const size_t capacity = JSON_OBJECT_SIZE(3) + JSON_ARRAY_SIZE(2) + 60;
      DynamicJsonDocument doc(capacity);
      DeserializationError error = deserializeJson(doc, client);
      if (error) {
        Serial.print(F("deserializeJson() failed: "));
        Serial.println(error.c_str());
        return;
       }
      } else {
        Serial.printf("[HTTPS] GET... failed, error: %s\n", https.errorToString(httpCode).c_str());
      }
 
      https.end();
    } else {
      Serial.printf("[HTTPS] Unable to connect\n");
    }
  }
  // Extract values
  Serial.println(F("Response:"));
  Serial.println("Wait 180s before next round...");
  delay(180000);
}
}

any direction that would help me get some usable output would be much appreciated.

-Jason

2

but I dont understand what to do with it..

I don't know why you did that, but it looks to me like that is an attempt to predict the size of the JSON data, so the resulting value would be assigned to capacity.

      DeserializationError error = deserializeJson(doc, client);

What did this do?

3

I commented out the DeserializationError stuff for the time being... now I'm getting a http code: 200 returned. so some progress :slight_smile: .

what is the proper syntax for printing the parsed data?

Serial.print(doc["dayOfWeek"]["Monday"]["narrative"].as<char*>());

returns 'doc' was not declared in this scope error. but isnt doc a few lines up at

DynamicJsonDocument doc(capacity);

-Jason

4

Assistant | ArduinoJson 6 shows Arduino code to extract fields from JSON strings. Page down to see the code.

The example code looks like this.

const size_t capacity = JSON_ARRAY_SIZE(2) + JSON_OBJECT_SIZE(3) + 30;
DynamicJsonDocument doc(capacity);

const char* json = "{\"sensor\":\"gps\",\"time\":1351824120,\"data\":[48.75608,2.302038]}";

deserializeJson(doc, json);

const char* sensor = doc["sensor"]; // "gps"
long time = doc["time"]; // 1351824120

float data_0 = doc["data"][0]; // 48.75608
float data_1 = doc["data"][1]; // 2.302038
5

yes I have looked at that as several examples, I still cannot follow... I have updated just about every line with Serial.printf output to try to show me whats happening... I'm still not seeing what Im missing.

-Jason

6

I'm still not seeing what Im missing.

What I see that you are missing is a clue as to what you are parsing.

Instead of expecting deserializeJson() to get the data from the client, you should get the data, and pass the data to deserializeJson() (or some other function). That way, you can print the data, to make sure you have something valid to parse.

7

well yes, thats why I'm here seeking a little help... If i knew exactly what I was doing, It would be done and I wouldn't be here asking questions to learn more...

-J

8

Jmeredith:
well yes, thats why I'm here seeking a little help... If i knew exactly what I was doing, It would be done and I wouldn't be here asking questions to learn more...

-J

If you were really trying to learn, you would not have jumped in the deep end of the pool wearing cement shoes without being able to swim.

Look at the methods that HTTPClient has. In addition to being able to get the http status, it can get the server response. There just might be a clue-by-four lurking there.

You are calling a function, deserializeJson(), but you have not shown us that function. If that is part of the ArduinoJson library, you need to look at what variations of that function exist. Look for one that lets you supply a string or String, like the one you got from the server. If you got anything that looks like JSON from the server, pass it to that function.

9

Ok I think I got it! for anyone else trying to parse specific parts of the Wunderground API (new https version)

/**
   BasicHTTPSClient.ino
    Created on: 20.08.2018
*/
#include <ArduinoJson.h>
#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClientSecureBearSSL.h>

// Current fingerprint for https://api.wunderground.com
const uint8_t fingerprint[20] = {0x5D, 0xC3, 0x85, 0xD6, 0x19, 0x22, 0x99, 0x2B, 0x7B, 0xBB, 0x87, 0x8F, 0xD6, 0x9E, 0xF2, 0xD5, 0x44, 0xDA, 0x32, 0x28};

ESP8266WiFiMulti WiFiMulti;

void setup() {

  Serial.begin(115200);
  Serial.println();
  Serial.println();
  Serial.println();

  for (uint8_t t = 4; t > 0; t--) {
    Serial.printf("[SETUP] WAIT %d...\n", t);
    Serial.flush();
    delay(1000);
  }

  WiFi.mode(WIFI_STA);
  WiFiMulti.addAP("----ssid----", "----password----");
}

void loop() {
  // wait for WiFi connection
  if ((WiFiMulti.run() == WL_CONNECTED)) {
    std::unique_ptr<BearSSL::WiFiClientSecure>client(new BearSSL::WiFiClientSecure);
    client->setFingerprint(fingerprint);
    HTTPClient https;
    Serial.print("[HTTPS] begin...\n");
    if (https.begin(*client, "https://api.weather.com/v3/wx/forecast/daily/5day?postalKey=78249:US&units=e&language=en-US&format=json&apiKey=----GET-YOUR-OWN-KEY----")) {
      Serial.print("[HTTPS] GET...\n");
      int httpCode = https.GET();
      if (httpCode > 0) {
        Serial.printf("[HTTPS] GET... code: %d\n", httpCode);
    if (httpCode == HTTP_CODE_OK || httpCode == HTTP_CODE_MOVED_PERMANENTLY) {
      Serial.printf("[HTTPS] httpCode:  %d\n" , httpCode);
      //pass GET results to doc
      String payload = https.getString();
      //print payload
      Serial.println(payload);
      // assign memory pool expression based on https://arduinojson.org/v6/assistant/
      const size_t capacity = JSON_ARRAY_SIZE(1) + 20*JSON_ARRAY_SIZE(6) + 27*JSON_ARRAY_SIZE(12) + JSON_OBJECT_SIZE(21) + JSON_OBJECT_SIZE(27);
      // print the capacity
      Serial.printf("[JSON] capacity: %d\n" , capacity);
      // set the capacity of 'doc' 
      DynamicJsonDocument doc(capacity);
      //Deserialize the payload
      deserializeJson(doc, payload);
      //DeserializationError error = deserializeJson(doc, client);
      //if (error) {
      //  Serial.print(F("deserializeJson() failed: "));
      //  Serial.println(error.c_str());
      //  return;
      Serial.println();
      Serial.println();
      Serial.println();
      Serial.print("Weather for ");
      Serial.print(doc["dayOfWeek"][0].as<char*>());
      Serial.print(": ");
      Serial.println(doc["narrative"][0].as<char*>());  
       }
      } else {
        Serial.printf("[HTTPS] GET... failed, error: %s\n", https.errorToString(httpCode).c_str());
      }
 
      https.end();
    } else {
      Serial.printf("[HTTPS] Unable to connect\n");
    }
  }
  // Extract values
 // Serial.print(doc["narrative"][0].as<char*>());  
  Serial.println("\n\n Wait 180s before next round...");
  delay(180000);
}

once you have the api returning data with a URL like this:

https://api.weather.com/v3/wx/forecast/daily/5day?postalKey=78249:US&units=e&language=en-US&format=json&apiKey=----GET-YOUR-OWN-KEY----

then you can cut/paste the results over here: http://json.parser.online.fr/

here you can pick out what data you want returned... I am using dayOfWeek & narrative like this:

Serial.print(doc["dayOfWeek"][0].as<char*>());
Serial.println(doc["narrative"][0].as<char*>());

hope this can help someone else... it took me a few hours of banging around to figure it out

1 Like
10

it took me a few hours of banging around to figure it out

Good on you for persevering and arriving at a solution.

I see that you did get the server response as a String (ugh!) so you could see what you were parsing. Always a good idea to know what you are dealing with, IMHO.