NodeMCU/ESP12N Watchdog Timer and large JSON download

[ardvino]

Hi All,

Happy New Year! :-)

I am using a NodeMCU 1.0 (ESP12N) to connect to an API and pull down a large JSON file for parsing and using.

The full JSON is around 12000 characters long. When I try to download even half that much, the watchdog timer kicks in and resets the device before it's finished the download. It fails with reason 4, which I think is the hardware WDT. If I set the device to only download the first 100 characters or so, it all works fine and the download finishes before the WDT timeout.

Does anyone know a way that I can increase the WDT timeout value or somehow periodically feed the watchdog whilst the download is in progress?

Many thanks in advance.

Below is my code (confidential values redacted), and below that the wdt crash response. If I change the value of char json[***] (line 108) above about 3000, I start to hit the wdt timeout. Sorry it's scrappy chop-about code. I'm quite new to this IoT stuff and in the steep bit of the learning curve!

Code: [Select]


#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <ESP8266WebServer.h>
#include <ESP8266HTTPClient.h>

/* Set these to your desired credentials. */
const char *ssid = "REDACTED";  //ENTER YOUR WIFI SETTINGS
const char *password = "REDACTED";

//Link to read data from API
//Web/Server address to read/write from
const char *host = "REDACTED";
const int httpsPort = 443;  //HTTPS= 443 and HTTP = 80

//SHA1 finger print of certificate use web browser to view and copy
const char fingerprint[] PROGMEM = "REDACTED";
//=======================================================================
//                    Power on setup
//=======================================================================

void setup() {
  delay(1000);
  Serial.begin(115200);
  WiFi.mode(WIFI_OFF);        //Prevents reconnection issue (taking too long to connect)
  delay(1000);
  WiFi.mode(WIFI_STA);        //Only Station No AP, This line hides the viewing of ESP as wifi hotspot

  WiFi.begin(ssid, password);     //Connect to your WiFi router
  Serial.println("");

  Serial.print("Connecting");
  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  //If connection successful show IP address in serial monitor
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());  //IP address assigned to your ESP
}

//=======================================================================
//                    Main Program Loop
//=======================================================================
void loop() {
  WiFiClientSecure httpsClient;    //Declare object of class WiFiClient

  Serial.println(host);

  Serial.printf("Using fingerprint '%s'\n", fingerprint);
  httpsClient.setFingerprint(fingerprint);
  httpsClient.setTimeout(15000); // 15 Seconds
  delay(1000);

  Serial.print("HTTPS Connecting");
  int r = 0; //retry counter
  while ((!httpsClient.connect(host, httpsPort)) && (r < 30)) {
    delay(100);
    Serial.print(".");
    r++;
  }
  if (r == 30) {
    Serial.println("Connection failed");
  }
  else {
    Serial.println("Connected to web");
  }

  String getData, Link;

  //GET Data
  Link = "REDACTED";

  Serial.print("requesting URL: ");
  Serial.println(host + Link);

  httpsClient.print(String("GET ") + Link + " HTTP/1.1\r\n" +
                    "Host: " + host + "\r\n" +
                    "Connection: close\r\n\r\n");

  Serial.println("request sent");

  while (httpsClient.connected()) {
    String line = httpsClient.readStringUntil('\n');
    if (line == "\r") {
      Serial.println("headers received");
      break;
    }
  }

  Serial.println("reply was:");
  Serial.println("==========");
  String line;

  while (httpsClient.available()) {
    line = httpsClient.readStringUntil('\n');  //Read Line by Line
    char json[8000];
    line.toCharArray(json, 8000);
    const char* json1 = json;
    Serial.println(json1); //Print response

  }
  Serial.println("==========");
  Serial.println("closing connection");

  delay(1000);
  delay(1000);
  delay(1000);
  delay(1000);
  delay(1000);
  delay(1000);
  delay(1000);
  delay(1000);
  delay(1000);
  delay(1000);
}
This code connects fine, but half way through the download, produces this error:

Code: [Select]

ets Jan  8 2013,rst cause:4, boot mode:(3,7)

wdt reset
load 0x4010f000, len 3584, room 16
tail 0
chksum 0xb0
csum 0xb0
v2843a5ac
~ld

[runaway_pancake]

The solution is in the implementation of
yield();

[ardvino]

Thanks. I did try putting yield(); in various places but unfortunately it didn't help.

I have since discovered the problem. The JSON is just too big, and trying to jam it into one string causes overflow. This overflow scenario subsequently triggers the watchdog timer (the chip is sat there staring in dismay while characters pour out of the overflowing variable).

The answer has been to parse the JSON data as it comes in - much more sensible than pulling the lot and picking over it afterwards. My code now uses .readStringUntil to recognise control characters in the data and pull the useful bits into separate variables.