Using JSON to parse OpenWeatherMap data (DUE)

Hi,

Could someone please give me advice on the questions down below? I am just overwhelmed at how many libraries and methods there are.

I am trying to build a simple weatherstation using a DUE, 7 inch TFT and an ESP-01 (ESP8266) on Serial3. I am accessing data from openweathermap
and can successfully download a 7 day forecast using One Call API. Data is below but has some of my own formatting ...mydt..and a COUNT for readability only . Please assume i have original JSON formatted data. For starters I want to graphically display the 8 day ( Current +7 days) forecast on the TFT. My issue is using a JSON parser.....

{"lat":51.325,"lon":-0.1957,"timezone":"Europe/London","timezone_offset":3600,"current":{"dt
0 {mydt":1634298439,"sunrise":1634279064,"sunset":1634317700,"temp":285.47,"feels_like":284.87,"pressure":1021,"humidity":81,"dew_point":282.31,"uvi":1.39,"clouds":40,"visibility":10000,"wind_speed":0.45,"wind_deg":46,"wind_gust":2.24,"weather":[{"id":802,"main":"Clouds","description":"scattered clouds","icon":"03d"}]},"daily":[{"dt
1 {mydt":1634295600,"sunrise":1634279064,"sunset":1634317700,"moonrise":1634313300,"moonset":1634255220,"moon_phase":0.33,"temp":{"day":285.22,"min":281.44,"max":286.3,"night":281.44,"eve":284.47,"morn":283.56},"feels_like":{"day":284.55,"night":280.46,"eve":283.22,"morn":283.11},"pressure":1021,"humidity":79,"dew_point":281.69,"wind_speed":4.23,"wind_deg":40,"wind_gust":10.24,"weather":[{"id":803,"main":"Clouds","description":"broken clouds","icon":"04d"}],"clouds":52,"pop":0,"uvi":1.81},{"dt
2 {mydt":1634382000,"sunrise":1634365566,"sunset":1634403972,"moonrise":1634400900,"moonset":1634346420,"moon_phase":0.36,"temp":{"day":285.27,"min":281.39,"max":286.18,"night":282.72,"eve":284.43,"morn":282.56},"feels_like":{"day":284.44,"night":281.9,"eve":283.68,"morn":282.08},"pressure":1022,"humidity":73,"dew_point":280.9,"wind_speed":3.48,"wind_deg":190,"wind_gust":5.85,"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04d"}],"clouds":97,"pop":0,"uvi":0.84},{"dt
3 {mydt":1634468400,"sunrise":1634452068,"sunset":1634490245,"moonrise":1634488260,"moonset":1634437440,"moon_phase":0.4,"temp":{"day":287.86,"min":282.4,"max":287.86,"night":284.32,"eve":286.12,"morn":283.36},"feels_like":{"day":287.34,"night":283.71,"eve":285.48,"morn":282.68},"pressure":1019,"humidity":75,"dew_point":283.9,"wind_speed":4.69,"wind_deg":206,"wind_gust":8.5,"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04d"}],"clouds":100,"pop":0,"uvi":0.21},{"dt
4 {mydt":1634554800,"sunrise":1634538570,"sunset":1634576520,"moonrise":1634575560,"moonset":1634528340,"moon_phase":0.43,"temp":{"day":288.05,"min":283.72,"max":288.05,"night":286.88,"eve":286.13,"morn":284.25},"feels_like":{"day":287.4,"night":286.76,"eve":285.89,"morn":283.79},"pressure":1018,"humidity":69,"dew_point":282.76,"wind_speed":6.49,"wind_deg":191,"wind_gust":14.05,"weather":[{"id":500,"main":"Rain","description":"light rain","icon":"10d"}],"clouds":100,"pop":0.96,"rain":1.8,"uvi":0.46},{"dt
5 {mydt":1634641200,"sunrise":1634625073,"sunset":1634662795,"moonrise":1634662740,"moonset":1634619120,"moon_phase":0.46,"temp":{"day":289.35,"min":287.63,"max":289.59,"night":289.01,"eve":288.59,"morn":289.09},"feels_like":{"day":289.35,"night":288.92,"eve":288.54,"morn":289.19},"pressure":1011,"humidity":89,"dew_point":287.89,"wind_speed":8.08,"wind_deg":201,"wind_gust":16.58,"weather":[{"id":500,"main":"Rain","description":"light rain","icon":"10d"}],"clouds":97,"pop":1,"rain":4.95,"uvi":1.49},{"dt
6 {mydt":1634727600,"sunrise":1634711575,"sunset":1634749072,"moonrise":1634749920,"moonset":1634709780,"moon_phase":0.5,"temp":{"day":283.49,"min":281.93,"max":288.21,"night":283.98,"eve":282.7,"morn":286.64},"feels_like":{"day":282.51,"night":283.31,"eve":278.73,"morn":286.34},"pressure":996,"humidity":74,"dew_point":279.51,"wind_speed":10.18,"wind_deg":265,"wind_gust":17.86,"weather":[{"id":501,"main":"Rain","description":"moderate rain","icon":"10d"}],"clouds":92,"pop":1,"rain":11.93,"uvi":2},{"dt
7 {mydt":1634814000,"sunrise":1634798079,"sunset":1634835349,"moonrise":1634837220,"moonset":1634800440,"moon_phase":0.53,"temp":{"day":287.51,"min":283.92,"max":287.51,"night":283.93,"eve":284.57,"morn":284.12},"feels_like":{"day":286.65,"night":283.1,"eve":283.78,"morn":283.41},"pressure":1006,"humidity":63,"dew_point":280.86,"wind_speed":8.04,"wind_deg":260,"wind_gust":15.19,"weather":[{"id":500,"main":"Rain","description":"light rain","icon":"10d"}],"clouds":50,"pop":0.39,"rain":0.46,"uvi":2},{"dt
8 {mydt":1634900400,"sunrise":1634884582,"sunset":1634921628,"moonrise":1634924700,"moonset":1634891160,"moon_phase":0.56,"temp":{"day":280.76,"min":280.05,"max":281.97,"night":280.05,"eve":280.06,"morn":280.38},"feels_like":{"day":279.77,"night":277.92,"eve":277.88,"morn":278.95},"pressure":1013,"humidity":86,"dew_point":278.83,"wind_speed":3.95,"wind_deg":308,"wind_gust":9.05,"weather":[{"id":500,"main":"Rain","description":"light rain","icon":"10d"}],"clouds":100,"pop":0.55,"rain":1.1,"uvi":2}]}
Bytes Rxd =4740

  1. Which JSON library should I use?
  2. The data is 4,740 bytes long. Can I use a JSON call to read the data as it comes in on Serial3, thus not having to create a string of 4,740 bytes length?
    This is the most confusing thing for me.
  3. Can someone point me an example where they using an Arduino board (MEGA/DUE)and ESP-01. ( not an example with a Main ESP board)?

I obviously want data like max temp, min temp, sunrise, sunset and weather forecast etc for each day .... from a JSON parse.

A full blown One Call API download is around 20,500 bytes, but I will start with just the current + 7 day forecast at 4700 bytes.

Thanks

Hi @cblx5,
I use the arduinoJson6.h library
ref: Documentation | ArduinoJson 6

I also use the website http://jsonviewer.stack.hu/
to see if my json format string is correct.

RV mineirin

Hi ruilviana,

I had looked at ArduinoJson 6 and will use that .

I did a full blown One Call API of 20,508 bytes and pasted it into the json viewer and it was good ( as expected). So i have the data.... I just need it in an array of some form in my DUE.

I am still unclear on question 2 above.... I have seen stuff that include error reporting, time outs and streaming....and then lost it :-))).

Many thanks

I haven't looked at JSON libraries lately, but I would expect that it would want a complete object to parse before you could access component parts. The Due has plenty of RAM though.

That said, I expect you could make several calls to the weather service and get smaller individual responses.

Hi wildbill,

Yes - I did start thinking I could use a MEGA board but saw that its 8K for RAM would be eaten up instantly hence I switched to the DUE at 96K RAM and about 5 times faster.

Yes I can make shorter calls to openweathermap, so call for 'minutely' data, then 'hourly', then 'daily'.

I just got the impression that some examples got data from the ESPxxxx module in realtime, rather then copy to very large string and then parse.

I do hope someone could give me a pointer?

Thanks

The ArduinoJson library has an assistant that will write your serialization and deserialization code for you:

I tried it on your JSON and it reported "Unexpected token in JSON at position 92". That's about at the '0' in the middle of:

"current":{"dt
0 {mydt":1634298439,

That bit doesn't look right to me.

Hi johnwasser,

my actual data is here

{"lat":51.325,"lon":-0.1957,"timezone":"Europe/London","timezone_offset":3600,"current":{"dt":1634305174,"sunrise":1634279064,"sunset":1634317700,"temp":286.7,"feels_like":285.99,"pressure":1021,"humidity":72,"dew_point":281.76,"uvi":1.31,"clouds":40,"visibility":10000,"wind_speed":0.45,"wind_deg":57,"wind_gust":2.68,"weather":[{"id":802,"main":"Clouds","description":"scattered clouds","icon":"03d"}]},"daily":[{"dt":1634295600,"sunrise":1634279064,"sunset":1634317700,"moonrise":1634313300,"moonset":1634255220,"moon_phase":0.33,"temp":{"day":285.4,"min":282.05,"max":286.7,"night":282.49,"eve":285.76,"morn":283.56},"feels_like":{"day":284.59,"night":281.79,"eve":284.77,"morn":283.11},"pressure":1021,"humidity":73,"dew_point":280.71,"wind_speed":4.13,"wind_deg":37,"wind_gust":10.24,"weather":[{"id":803,"main":"Clouds","description":"broken clouds","icon":"04d"}],"clouds":76,"pop":0.1,"uvi":1.81},{"dt":1634382000,"sunrise":1634365566,"sunset":1634403972,"moonrise":1634400900,"moonset":1634346420,"moon_phase":0.36,"temp":{"day":286.07,"min":282.74,"max":287.13,"night":282.74,"eve":285.06,"morn":282.99},"feels_like":{"day":285.27,"night":281.83,"eve":284.45,"morn":282.36},"pressure":1021,"humidity":71,"dew_point":281.28,"wind_speed":3.57,"wind_deg":180,"wind_gust":5.56,"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04d"}],"clouds":96,"pop":0.14,"uvi":0.84},{"dt":1634468400,"sunrise":1634452068,"sunset":1634490245,"moonrise":1634488260,"moonset":1634437440,"moon_phase":0.4,"temp":{"day":287.88,"min":282.57,"max":288.58,"night":283.54,"eve":285.8,"morn":283.04},"feels_like":{"day":287.29,"night":282.91,"eve":285.18,"morn":282.16},"pressure":1018,"humidity":72,"dew_point":283.15,"wind_speed":4.74,"wind_deg":194,"wind_gust":8.11,"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04d"}],"clouds":100,"pop":0,"uvi":0.21},{"dt":1634554800,"sunrise":1634538570,"sunset":1634576520,"moonrise":1634575560,"moonset":1634528340,"moon_phase":0.43,"temp":{"day":288.93,"min":283.31,"max":288.93,"night":288.41,"eve":287.97,"morn":283.69},"feels_like":{"day":288.44,"night":288.24,"eve":287.54,"morn":283.25},"pressure":1018,"humidity":72,"dew_point":284.32,"wind_speed":6.82,"wind_deg":191,"wind_gust":13.83,"weather":[{"id":500,"main":"Rain","description":"light rain","icon":"10d"}],"clouds":100,"pop":0.54,"rain":0.35,"uvi":0.46},{"dt":1634641200,"sunrise":1634625073,"sunset":1634662795,"moonrise":1634662740,"moonset":1634619120,"moon_phase":0.46,"temp":{"day":289.72,"min":288.21,"max":289.91,"night":289.91,"eve":289.77,"morn":288.81},"feels_like":{"day":289.83,"night":289.68,"eve":289.65,"morn":288.81},"pressure":1014,"humidity":92,"dew_point":288.74,"wind_speed":7.87,"wind_deg":215,"wind_gust":15.89,"weather":[{"id":500,"main":"Rain","description":"light rain","icon":"10d"}],"clouds":100,"pop":1,"rain":3.58,"uvi":1.49},{"dt":1634727600,"sunrise":1634711575,"sunset":1634749072,"moonrise":1634749920,"moonset":1634709780,"moon_phase":0.5,"temp":{"day":284.79,"min":284.36,"max":289.57,"night":284.93,"eve":284.36,"morn":286.84},"feels_like":{"day":283.73,"night":284.17,"eve":283.68,"morn":286.54},"pressure":996,"humidity":66,"dew_point":279.07,"wind_speed":10.42,"wind_deg":277,"wind_gust":18.66,"weather":[{"id":501,"main":"Rain","description":"moderate rain","icon":"10d"}],"clouds":53,"pop":1,"rain":8.13,"uvi":2},{"dt":1634814000,"sunrise":1634798079,"sunset":1634835349,"moonrise":1634837220,"moonset":1634800440,"moon_phase":0.53,"temp":{"day":285.67,"min":283.18,"max":286.49,"night":284.52,"eve":283.97,"morn":283.18},"feels_like":{"day":284.91,"night":283.8,"eve":283.07,"morn":282.28},"pressure":1002,"humidity":74,"dew_point":281.48,"wind_speed":10.64,"wind_deg":274,"wind_gust":18.52,"weather":[{"id":500,"main":"Rain","description":"light rain","icon":"10d"}],"clouds":43,"pop":0.64,"rain":1.11,"uvi":2},{"dt":1634900400,"sunrise":1634884582,"sunset":1634921628,"moonrise":1634924700,"moonset":1634891160,"moon_phase":0.56,"temp":{"day":283.3,"min":279.45,"max":283.3,"night":279.45,"eve":280.86,"morn":280.74},"feels_like":{"day":281.7,"night":277.14,"eve":277.97,"morn":277.08},"pressure":1012,"humidity":51,"dew_point":274.01,"wind_speed":8.6,"wind_deg":276,"wind_gust":15.98,"weather":[{"id":500,"main":"Rain","description":"light rain","icon":"10d"}],"clouds":17,"pop":0.75,"rain":0.64,"uvi":2}]}

A had put in some extra characters to make reading easier, as I stated in my first post.

thanks for the 'Assistant' tip I will try that, it took me a few hours (learning)
to work out to get morning temperature on day 7 needed.....

float dtemp = doc["daily"][7]["temp"]["morn"];

Seems so obvious now.

Thanks for tip.

Hi, thanks for you help guys.... here is a early version using semi-live data from Openweather.

2 Likes

Hi @cblx5
Very beautiful your project. Congratulations

Can you publish your sketch?

RV mineirin

Turns out that this isn't true - the library I finally checked on had a function that can parse a subset of an object, so it's possible to stream characters from { to } and use the library to pull pieces by name.

Hi RV,

The code and hardware details are from here......

ESP8266 Weather Widget V2.0 : 13 Steps (with Pictures) - Instructables

and the code is on GitHub here....

GitHub - ThingPulse/esp8266-weather-station-color: ESP8266 Weather Station in Color using ILI9341 TFT 240x320 display

All I have done is to modify it for my TFT (my TFT is wider so did more days) and MEGA. I do not have 'touch' capability and do not not have a fully working sketch. I have 3 very large sketch's based on library examples. one to download the data, one to parse it, one to display parsed data as per my photo.

1 Like

Tks