Help with backtraces and crashes

I have created a complex (at least for me) little auto gauge that uses an ESP32S3 Dev board, LVGL, CAN BUS, GPS, TPMS, Bluetooth and Wifi.

It all works well save for some errors and crashes when I switch from one of the screens.

Some background: the gauge has 8 screens, and on all but one, BLE is active and using the antenna. On the other screen, BLE gets deactivated and WiFi is activated so that the user can log into the gauge to set some preferences as there's no user input directly with the gauge.
The ESP shares the antenna between WiFi and BLE and I need both to be available for different functions (BLE for the TPMS and WiFi for an access point to set preferences) so that's why the screen switches which radio gets access to the antenna.

When I switch screens from the WiFi-active screen, it crashes and freezes and I have to reboot the board.

Here's what is output on the Serial Monitor when the crashes happen:


E (337393) BLE_INIT: Malloc failed
assert emi.c 164, param 00000000 00008800
Guru Meditation Error: Core  0 panic'ed (Interrupt wdt timeout on CPU0). 

Core  0 register dump:
PC      : 0x40006fcb  PS      : 0x00060034  A0      : 0x8000a3d8  A1      : 0x3fcc6bf0  
A2      : 0x00000000  A3      : 0x00008800  A4      : 0x3ff196ce  A5      : 0x000000a4  
A6      : 0x00060e23  A7      : 0x00000000  A8      : 0x00000001  A9      : 0x3fcef9e4  
A10     : 0x3fcef9e4  A11     : 0x00000001  A12     : 0x00000000  A13     : 0x00008800  
A14     : 0x00000001  A15     : 0x00ffffff  SAR     : 0x00000005  EXCCAUSE: 0x00000005  
EXCVADDR: 0x00000000  LBEG    : 0x40006fc4  LEND    : 0x40006fcb  LCOUNT  : 0x00000000  


Backtrace: 0x40006fc8:0x3fcc6bf0 0x4000a3d5:0x3fcc6c10 0x40007303:0x3fcc6c30 0x420ca757:0x3fcc6c50 0x420cae1e:0x3fcc6c90 0x420d198f:0x3fcc6cd0 0x420d20b1:0x3fcc6d00 0x420c7f4c:0x3fcc6d20 0x40379c68:0x3fcc6d40


Core  1 register dump:
PC      : 0x42129756  PS      : 0x00060d34  A0      : 0x8205367c  A1      : 0x3fcf4fd0  
A2      : 0x00000000  A3      : 0x00000000  A4      : 0x00060520  A5      : 0x3fceba80  
A6      : 0x02c9a4c0  A7      : 0x00ffffff  A8      : 0x00000000  A9      : 0x00000004  
A10     : 0x3fcf518c  A11     : 0x80000001  A12     : 0x803836a5  A13     : 0x3fceba80  
A14     : 0x00060023  A15     : 0x00000003  SAR     : 0x00000000  EXCCAUSE: 0x00000005  
EXCVADDR: 0x00000000  LBEG    : 0x00000000  LEND    : 0x00000000  LCOUNT  : 0x00000000  


Backtrace: 0x42129753:0x3fcf4fd0 0x42053679:0x3fcf4ff0 0x4038184c:0x3fcf5010




ELF file SHA256: ce21a5c3e908bdbf

Guru Meditation Error: Core  0 panic'ed (Interrupt wdt timeout on CPU0). 

Core  0 register dump:
PC      : 0x4038866f  PS      : 0x00020534  A0      : 0x820d3bfa  A1      : 0x3fcc6a00  
A2      : 0x00020523  A3      : 0xa5a5a5a5  A4      : 0x820d3bfa  A5      : 0x00060525  
A6      : 0xfffbfff0  A7      : 0x00000046  A8      : 0x3fc9ba8c  A9      : 0x00000001  
A10     : 0x60000000  A11     : 0x00000001  A12     : 0x0000000a  A13     : 0x3fcc69ec  
A14     : 0x00000001  A15     : 0x3fcf5010  SAR     : 0x0000000a  EXCCAUSE: 0x00000005  
EXCVADDR: 0x00000000  LBEG    : 0x400570e8  LEND    : 0x400570f3  LCOUNT  : 0x00000000  


Backtrace: 0x4038866c:0x3fcc6a00 0x420d3bf7:0x3fcc6a10 0x420d4205:0x3fcc6a30 0x42054469:0x3fcc6a50 0x4205479a:0x3fcc6ac0 0x40377b81:0x3fcc6b10 0x40377538:0x3fcc6b30 0x00040022:0x3fcc6bf0 |<-CORRUPTED


Core  1 register dump:
PC      : 0x42129756  PS      : 0x00060d34  A0      : 0x8205367c  A1      : 0x3fcf4fd0  
A2      : 0x00000000  A3      : 0x00000000  A4      : 0x00060520  A5      : 0x3fceba80  
A6      : 0x02c9a4c0  A7      : 0x00ffffff  A8      : 0x00000000  A9      : 0x00000004  
A10     : 0x3fcf518c  A11     : 0x80000001  A12     : 0x803836a5  A13     : 0x3fceba80  
A14     : 0x00060023  A15     : 0x00000003  SAR     : 0x00000000  EXCCAUSE: 0x00000005  
EXCVADDR: 0x00000000  LBEG    : 0x00000000  LEND    : 0x00000000  LCOUNT  : 0x00000000  


Backtrace: 0x42129753:0x3fcf4fd0 0x42053679:0x3fcf4ff0 0x4038184c:0x3fcf5010




ELF file SHA256: ce21a5c3e908bdbf

Re-entered core dump! Exception happened during core dump!
Rebooting...
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40377720
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x4bc
load:0x403c9700,len:0xbd8
load:0x403cc700,len:0x2a0c
entry 0x403c98d0

Here's the results of the backtrace:


Backtrace: 0x40006fc8:0x3fcc6bf0 0x4000a3d5:0x3fcc6c10 0x40007303:0x3fcc6c30 0x420ca757:0x3fcc6c50 0x420cae1e:0x3fcc6c90 0x420d198f:0x3fcc6cd0 0x420d20b1:0x3fcc6d00 0x420c7f4c:0x3fcc6d20 0x40379c68:0x3fcc6d40

No output

—

Backtrace: 0x42129753:0x3fcf4fd0 0x42053679:0x3fcf4ff0 0x4038184c:0x3fcf5010

Decoding stack results

0x42129753: esp_pm_impl_waiti at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/hal/esp32s3/include/hal/cpu_ll.h:182

0x42053679: esp_vApplicationIdleHook at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/freertos_hooks.c:63

0x4038184c: prvIdleTask at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/tasks.c:4099

—

Backtrace: 0x4038866c:0x3fcc6a00 0x420d3bf7:0x3fcc6a10 0x420d4205:0x3fcc6a30 0x42054469:0x3fcc6a50 0x4205479a:0x3fcc6ac0 0x40377b81:0x3fcc6b10 0x40377538:0x3fcc6b30 0x00040022:0x3fcc6bf0 |<-CORRUPTED

Decoding stack results

0x420d3bf7: esp_core_dump_write at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/espcoredump/include_core_dump/port/xtensa/esp_core_dump_port_impl.h:67

0x420d4205: esp_core_dump_to_flash at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/espcoredump/src/core_dump_flash.c:342

0x42054469: esp_panic_handler at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:352

0x4205479a: panic_handler at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/panic_handler.c:188

0x40377b81: panicHandler at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/panic_handler.c:213

—

Backtrace: 0x42129753:0x3fcf4fd0 0x42053679:0x3fcf4ff0 0x4038184c:0x3fcf5010

Decoding stack results

0x42129753: esp_pm_impl_waiti at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/hal/esp32s3/include/hal/cpu_ll.h:182

0x42053679: esp_vApplicationIdleHook at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/freertos_hooks.c:63

0x4038184c: prvIdleTask at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/tasks.c:4099

I can usually figure out what's going on but this feels really different and I'd appreciate any insights as to what's going on and how to resolve it. I'm happy to post the code, it's 2000 lines long though so I don't know if that's helpful or not...

You already notice this error:

As a result, the WDT seems to be firing in core 0.

I think the memory map is different for each MCU in the ESP32 family, but using BT reserves memory for the controller and reduces heap space.

Therefore, check your memory usage, especially your heap memory usage.

Reference:

DRAM Organisation

DRAM Organisation with BT Enabled

Thanks, I ended up figuring it out after a bit of digging. Turns out that when I was deactivating BLE I was clearing it from the heap and everywhere else thinking that would free memory as that seemed to be the problem. Yet when I went to reactivate it we had crashes.

I was doing this:

BLEDevice::deinit(true);  
btStop();           

and later this:

btStart(); 

By switching to:

BLEDevice::deinit(false);  
btStop();                 

The problem was solved. No more panics and crashes = happy camper!

1 Like