Esp8266(D1 mini) sometimes reset by watchdog during http.get

sadfiesch · December 27, 2022, 9:38am

Hi Mates

I am currently developing a simple program that gets data from coingecko api and displays it on a tft display using tft_espi.

Everything works. However, watchdog sometimes resets my eps during the http.get method. I'm going crazy trying to fix this issue as everything i googled doesn't seem to work.

Here is what i've tried:

Add yield() to everything
Check free heap memory all the time(dont think its a memory issue as the get request sometimes works with the amout of memory and sometimes crashes with the same amout of memory)
Changed the esp
Changed the powercable

Here is my code:

void getChartDataFromApi(String ids, String vs_curry, String days, String interval){
  Serial.println("started shit method");
  if(WiFi.status() == WL_CONNECTED){
    String host = "https://api.coingecko.com/api/v3/coins/" + ids + "/market_chart?vs_currency=" + vs_curry + "&days=" + days + "&interval=" + interval;
    WiFiClientSecure chartClient;
    HTTPClient chartHttp;
    Serial.println("Setup 1");
    chartClient.setInsecure(); //the magic line, use with caution
    chartClient.connect(host, 443);
    Serial.println("After client connect");
    //chartClient.setTimeout(10);

    chartHttp.begin(chartClient, host);
    chartHttp.useHTTP10(true);
    Serial.println("After http begin");
    //chartHttp.setTimeout(10);

    Serial.print("free memory before get: "); 
    Serial.println(ESP.getFreeHeap(),DEC);
    yield();
    int http_code = chartHttp.GET();
    Serial.print("free memory after get: "); 
    Serial.println(ESP.getFreeHeap(),DEC);

    Serial.println("http code: " || String(http_code));
    if (http_code == HTTP_CODE_OK){
      Serial.println("after get");
      DynamicJsonDocument filter(16);
      filter["prices"] = true;

      DynamicJsonDocument doc(1024);
      Serial.println("after creation of doc");
      DeserializationError error = deserializeJson(doc, chartHttp.getStream(), DeserializationOption::Filter(filter));
      Serial.println("after deserialization");
      if (error) {
        Serial.print(F("deserializeJson() failed: "));
        Serial.println(error.f_str());
        return;
      }
      Serial.println("after error if");
      JsonArray prices = doc["prices"];
      Serial.println("Saving prices");
      //todo make this better
      //Dont judge me this is temporary :)
      currentChartPrices[0] = prices[0][1].as<float>(); // 16422.562273279298
      currentChartPrices[1] = prices[1][1].as<float>(); // 16899.331969046758
      currentChartPrices[2] = prices[2][1].as<float>(); // 16816.85370118997
      currentChartPrices[3] = prices[3][1].as<float>(); // 16824.790162441503
      currentChartPrices[4] = prices[4][1].as<float>(); // 16791.45543916491
      currentChartPrices[5] = prices[5][1].as<float>(); // 16848.568708323422
      currentChartPrices[6] = prices[6][1].as<float>(); // 16842.177039905502
      currentChartPrices[7] = prices[7][1].as<float>(); // 16838.576336989227

      Serial.println("after saving prices");
      chartHttp.end();
      chartClient.stop();
      filter.clear();
      doc.clear();
      Serial.println("after closing everything");
    }
    else{
      Serial.println("Failed to get chart api data");
      delay(300);
    }
  }
  Serial.println("outside of if");
}

Decoded exception including last few println that could be usfull as they show free heap memory:

Starting setup
free memory before setup: 46368
Setting up display
Done setting up display
Getting wifi creds from flash memory
Done getting wifi creds from flash memory
Getting Wifi Creds
Done getting Wifi Creds
Connecting to wifi
Done connecting to wifi
Setting up settings server
Starting settings server
Settings server started
done setting up settings server
free memory after setup: 44160
Loopcounter: 1
Free Memory: 44160
Symbol: btc
Price: 16917.34 Change: 0.46
Free Memory before chart data: 43960
started shit method
Setup 1
After client connect
After http begin
free memory before get: 37224
free memory after get: 15688
1
after get
after creation of doc
after deserialization
after error if
Saving prices
after saving prices
after closing everything
outside of if
Price: 16899.33
Price: 16816.85
Price: 16824.79
Price: 16791.46
Price: 16848.57
Price: 16842.18
Price: 16908.65
Free Memory after chart data: 43632
Y: 116
X: 0
X: 46.00 Y: 127.95 Price: 16899.33
X: 77.00 Y: 198.33 Price: 16816.85
X: 108.00 Y: 191.55 Price: 16824.79
X: 139.00 Y: 220.00 Price: 16791.46
X: 170.00 Y: 171.26 Price: 16848.57
X: 201.00 Y: 176.72 Price: 16842.18
X: 232.00 Y: 120.00 Price: 16908.65
Loopcounter: 2
Free Memory: 43960
Symbol: btc
Price: 16917.34 Change: 0.46
Free Memory before chart data: 43768
started shit method
Setup 1
After client connect
After http begin
free memory before get: 37032
free memory after get: 15496
1
after get
after creation of doc
after deserialization
after error if
Saving prices
after saving prices
after closing everything
outside of if
Price: 16899.33
Price: 16816.85
Price: 16824.79
Price: 16791.46
Price: 16848.57
Price: 16842.18
Price: 16908.65
Free Memory after chart data: 43440
Y: 116
X: 0
X: 46.00 Y: 127.95 Price: 16899.33
X: 77.00 Y: 198.33 Price: 16816.85
X: 108.00 Y: 191.55 Price: 16824.79
X: 139.00 Y: 220.00 Price: 16791.46
X: 170.00 Y: 171.26 Price: 16848.57
X: 201.00 Y: 176.72 Price: 16842.18
X: 232.00 Y: 120.00 Price: 16908.65
Loopcounter: 3
Free Memory: 43768
Symbol: btc
Price: 16917.34 Change: 0.46
Free Memory before chart data: 43576
started shit method
Setup 1
After client connect
After http begin
free memory before get: 36840

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Soft WDT reset

>>>stack>>>

ctx: sys
sp: 3ffff490 end: 3fffffb0 offset: 01a0

0x40223d56 in optimistic_yield at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/core_esp8266_main.cpp:150 (discriminator 1)
0x40215675 in WiFiClient::available() at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266WiFi\src/WiFiClient.cpp:252
0x402175ef in BearSSL::WiFiClientSecureCtx::_run_until(unsigned int, bool) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266WiFi\src/WiFiClientSecureBearSSL.cpp:548
0x40249e2a in br_ssl_engine_hs_reset at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/ssl/bearssl/src/ssl/ssl_engine.c:1320
0x40217678 in BearSSL::WiFiClientSecureCtx::_wait_for_handshake() at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266WiFi\src/WiFiClientSecureBearSSL.cpp:585
0x402179bd in BearSSL::WiFiClientSecureCtx::_connectSSL(char const*) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266WiFi\src/WiFiClientSecureBearSSL.cpp:1170
0x402433bf in ip_chksum_pseudo at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/lwip2/builder/lwip2-src/src/core/inet_chksum.c:395
0x4023ea2e in tcp_output at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/lwip2/builder/lwip2-src/src/core/tcp_out.c:1361
0x4010601d in ets_timer_arm_new at ??:?
0x4022463a in __delay at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/core_esp8266_wiring.cpp:54
0x40215c9a in ClientContext::connect(ip4_addr*, unsigned short) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266WiFi\src/include/ClientContext.h:148
 (inlined by) WiFiClient::connect(IPAddress, unsigned short) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266WiFi\src/WiFiClient.cpp:162
0x40217b4c in BearSSL::WiFiClientSecureCtx::connect(char const*, unsigned short) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266WiFi\src/WiFiClientSecureBearSSL.cpp:232
0x40233140 in std::function<void (void*, AsyncClient*, unsigned int)>::function(std::function<void (void*, AsyncClient*, unsigned int)> const&) at ??:?
0x40233140 in std::function<void (void*, AsyncClient*, unsigned int)>::function(std::function<void (void*, AsyncClient*, unsigned int)> const&) at ??:?
0x40230c8e in BearSSL::WiFiClientSecure::connect(char const*, unsigned short) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266WiFi\src/WiFiClientSecureBearSSL.h:245
0x402206e8 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:191
0x4021f02f in HTTPClient::connect() at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266HTTPClient\src/ESP8266HTTPClient.cpp:867
0x40224645 in __delay at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/core_esp8266_wiring.cpp:57
0x4021fa83 in HTTPClient::sendRequest(char const*, unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266HTTPClient\src/ESP8266HTTPClient.cpp:436
0x402265ad in uart_write at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/uart.cpp:544
0x402265ad in uart_write at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/uart.cpp:544
0x402206e8 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:191
0x402206f4 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:194
0x40100458 in ets_post at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/core_esp8266_main.cpp:181
0x4021fc4e in HTTPClient::GET() at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\libraries\ESP8266HTTPClient\src/ESP8266HTTPClient.cpp:356
0x4020bc03 in getChartDataFromApi(String, String, String, String) at src/main.cpp:370
0x40240fd8 in etharp_output_LWIP2 at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/lwip2/builder/lwip2-src/src/core/ipv4/etharp.c:885
0x40100571 in millis at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/core_esp8266_wiring.cpp:188
0x402427d0 in ip4_output_if_opt_src at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/lwip2/builder/lwip2-src/src/core/ipv4/ip4.c:1765
0x40100b39 in umm_free_core at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:549
0x40100d5d in malloc at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:821
0x40100b39 in umm_free_core at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:549
0x40100b39 in umm_free_core at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:549
0x40100b39 in umm_free_core at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:549
0x40100d26 in free at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:595
0x4023aa28 in operator delete(void*) at /workdir/repo/gcc-gnu/libstdc++-v3/libsupc++/del_op.cc:50
0x40100b39 in umm_free_core at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:549
0x40100d26 in free at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:595
0x402220f4 in String::init() at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/WString.h:289
 (inlined by) String::invalidate() at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/WString.cpp:147
0x4020d9d8 in getPriceFromAPI(String, String) at src/main.cpp:302
0x40100b39 in umm_free_core at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:549
0x40220073 in EspClass::flashWrite(unsigned int, unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/Esp.cpp:884
0x402265ad in uart_write at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/uart.cpp:544
0x402206e8 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:191
0x402206f4 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:194
0x402206e8 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:191
0x40221718 in Print::write(char const*) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/Print.h:59
 (inlined by) unsigned int Print::printNumber<unsigned long>(unsigned long, unsigned char) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/Print.cpp:264
0x40232b30 in std::function<void (void*, AsyncClient*, unsigned int)>::function(std::function<void (void*, AsyncClient*, unsigned int)> const&) at ??:?
0x402206e8 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:191
0x402206f4 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:194
0x402206e8 in HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/HardwareSerial.h:191
0x402223e9 in String::copy(char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/WString.cpp:226
0x4020fbb3 in String::~String() at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/WString.h:79
 (inlined by) loop at src/main.cpp:731
0x40210da4 in TFT_eSPI::drawChar(unsigned short, int, int, unsigned char) at .pio\libdeps\d1_mini\TFT_eSPI/TFT_eSPI.cpp:4624
0x40222880 in String::isSSO() const at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/WString.h:262
 (inlined by) String::setLen(int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/WString.h:267
 (inlined by) String::concat(char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/WString.cpp:309
0x40100b39 in umm_free_core at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266\umm_malloc/umm_malloc.cpp:549
0x40213d00 in TFT_eSPI::drawString(char const*, int, int, unsigned char) at .pio\libdeps\d1_mini\TFT_eSPI/TFT_eSPI.cpp:5066
0x40213d00 in TFT_eSPI::drawString(char const*, int, int, unsigned char) at .pio\libdeps\d1_mini\TFT_eSPI/TFT_eSPI.cpp:5066
0x402265ad in uart_write at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/uart.cpp:544
0x40220036 in EspClass::flashWrite(unsigned int, unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/Esp.cpp:834
0x40220036 in EspClass::flashWrite(unsigned int, unsigned char const*, unsigned int) at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/Esp.cpp:834
0x40223da2 in loop_wrapper() at C:\Users\Sadfiesch\.platformio\packages\framework-arduinoespressif8266\cores\esp8266/core_esp8266_main.cpp:201

Edit:
Removed full code as to not be a bad example for googling people

Solution:
Caused a bufferoverflow by calling array[arraysize+1];

horace · December 27, 2022, 10:19am

can you post the complete code?
check you are not overrunning array bounds

sadfiesch · December 27, 2022, 10:29am

Uploaded.

OMG never thought about running out of bounds on the json doc as the request might come back with less values then expected let me update the code and check.

good idea

sadfiesch · December 27, 2022, 10:40am

Its not the jsondoc running out of bounds it crashes before that.

horace · December 27, 2022, 10:45am

you define

float currentChartPrices[7];

which has elements currentChartPrice[0] to currentChartPrice[6]
and access currentChartPrices[0] to currentChartPrices[7] overrunning the array

//Dont judge me this is temporary :)
      currentChartPrices[0] = prices[0][1].as<float>(); // 16422.562273279298
      currentChartPrices[1] = prices[1][1].as<float>(); // 16899.331969046758
      currentChartPrices[2] = prices[2][1].as<float>(); // 16816.85370118997
      currentChartPrices[3] = prices[3][1].as<float>(); // 16824.790162441503
      currentChartPrices[4] = prices[4][1].as<float>(); // 16791.45543916491
      currentChartPrices[5] = prices[5][1].as<float>(); // 16848.568708323422
      currentChartPrices[6] = prices[6][1].as<float>(); // 16842.177039905502
      currentChartPrices[7] = prices[7][1].as<float>(); // 16838.576336989227

make the array size

float currentChartPrices[8];

sadfiesch · December 27, 2022, 10:47am

omg what a noobie mistake
Let me quickly change that and test.

But wouldn't that make the code crash everytime? how come it works 70% of the time?

horace · December 27, 2022, 10:49am

it depends on what the overrun corrupts
in practice code can run for years and then on a slightly different data set crash
in C++ one can use std::vector which throws an exception if bounds are overrun
in Java arrays have builtin array bounds checking
however, array bounds checking has a run time overhead

sadfiesch · December 27, 2022, 10:51am

Adjusted the code and trying to get the error again sometimes took quite a while.
If this was it i think its time to stop coding and get into gardening or something else

Thanks for your help in advanced.

horace · December 27, 2022, 10:59am

what the prices[][] array?

sadfiesch · December 27, 2022, 11:01am

its hystoric market data coming from coingecko

sadfiesch · December 27, 2022, 11:03am

Thanks a lot for your help

system · June 25, 2023, 11:03am

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
2 boards - mutual hardware watchdog Programming	7	342	April 9, 2023
Watchdog reset while downloading/saving SPIFFS (ESP32) Programming	11	123	November 29, 2024
Tests and feedback on new watchdog feature Cloud IoT	9	1725	November 18, 2021
ESP8266 Client connection causing restart? Programming	11	3723	May 6, 2021
Why does HttpClient crash? General Guidance	5	681	May 5, 2021

Esp8266(D1 mini) sometimes reset by watchdog during http.get

Related topics