ESP32 How To Generate Map File?

My ESP32 app is throwing an exception somewhat randomly - a "Guru Meditation Error", due to an illegal memory access. Seems like the only way to figure out where it's going wrong is to find out where the PC is in the code when the exception occurs, which requires a Map file. I have been totally unable to figure out how to generate a map file during the build, or from the elf file.

Anyone know how to generate a map file? I've Googled, found lots of articles, but all seem to be for older versions of Arduino and/or gcc, and none have worked.

Have you tried to use the EspExceptionDecoder?

See also this doc

The exception decoder does not give any useful information. There is not a full exception dump as happens for some exceptions - only a short register and stack dump.

I did finally figure out how to get the Map file: add -Xlinker -Map=arduino.map to the linker command line in platform.txt. At the time of the exception it appears to be inside a FastLED interrupt handler. I am now working my way through FastLED to figure out exactly where it is, and what is happening.

Ok
Would be good to share your experience, I’ve never played in depth with the core dumps capabilities of the esp32.

The exception dump shows only this:

Guru Meditation Error: Core  1 panic'ed (Interrupt wdt timeout on CPU1)
Core 1 register dump:
PC      : 0x400ea16b  PS      : 0x00050034  A0      : 0x400853c8  A1      : 0x3ffbe7a0
A2      : 0x00000007  A3      : 0x3ff56000  A4      : 0x00000000  A5      : 0x3ffc2758
A6      : 0x00000001  A7      : 0x00000001  A8      : 0x0000001e  A9      : 0x3ffb1e9d
A10     : 0x00000000  A11     : 0x00000009  A12     : 0x80182cbd  A13     : 0x3ffb1df0
A14     : 0x3ffc18dc  A15     : 0x3ffb2748  SAR     : 0x0000001a  EXCCAUSE: 0x00000006
EXCVADDR: 0x00000000  LBEG    : 0x400e5f5d  LEND    : 0x400e5f6c  LCOUNT  : 0x00000000
Core 1 was running in ISR context:
EPC1    : 0x400e6e6a  EPC2    : 0x00000000  EPC3    : 0x00000000  EPC4    : 0x400ea16b

Backtrace: 0x400ea16b:0x3ffbe7a0 0x400853c5:0x3ffbe7d0 0x400e6e67:0x3ffb1e30 0x400e6e8d:0x3ffb1e50 0x400e6ead:0x3ffb1e70 0x400e6f39:0x3ffb1e90 0x400e6f67:0x3ffb1ec0 0x400d71f2:0x3ffb1ee0 0x400d6fdd:0x3ffb1f00 0x400d7045:0x3ffb1f20 0x400e7547:0x3ffb1f40 0x400d5305:0x3ffb1f70 0x400d54db:0x3ffb1f90 0x400f06d1:0x3ffb1fb0 0x400897d1:0x3ffb1fd0

Core 0 register dump:
PC      : 0x4008767d  PS      : 0x00060f34  A0      : 0x800837e6  A1      : 0x3ffb9d80
A2      : 0x00000000  A3      : 0x00000001  A4      : 0x3ffb7da0  A5      : 0x00000000
A6      : 0x00000000  A7      : 0x00000002  A8      : 0x00000000  A9      : 0x00000001
A10     : 0x00000000  A11     : 0x13000249  A12     : 0x00000018  A13     : 0x00000001
A14     : 0x00060520  A15     : 0x00000000  SAR     : 0x00000000  EXCCAUSE: 0x00000006
EXCVADDR: 0x00000000  LBEG    : 0x00000000  LEND    : 0x00000000  LCOUNT  : 0x00000000

Backtrace: 0x4008767d:0x3ffb9d80 0x400837e3:0x3ffb9da0 0x400897d1:0x3ffb9dc0

Interesting, since all the other times I looked at it, the exception was an illegal memory exception, with the offending address of either 0x00000000, or 0xFFFFFFFF. But this time it is a watchdog timeout.

The Core1 PC is 0x400EA16B, which appears to be pointing into the middle of the FastLED showPixels function in clockless_block_ESP32.h, based on this entry in the map file:

 .text._ZN19ClocklessControllerILi2ELi60ELi150ELi90EL6EOrder66ELi0ELb0ELi5EE16interruptHandlerEPv
                0x400ea0bc       0xb9 E:\Users\RayL\Documents\Arduino\Build\sketch\src\GaugeBase\MasterGauge.cpp.o
                                 0xc0 (size before relaxing)
                0x400ea0bc                _ZN19ClocklessControllerILi2ELi60ELi150ELi90EL6EOrder66ELi0ELb0ELi5EE16interruptHandlerEPv
 *fill*         0x400ea175        0x3

The Core0 PC is 0x4008767d, which appears to be in iRAM, associated with spi_flash_op_block_func, whatever that is, based on this entry in the map file:

 .iram1         0x400875d4      0x348 C:\Users\RayL\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4/tools/sdk/lib\libspi_flash.a(cache_utils.o)
                                0x378 (size before relaxing)
                0x40087648                spi_flash_op_block_func
                0x40087708                spi_flash_disable_interrupts_caches_and_other_cpu
                0x40087804                spi_flash_enable_interrupts_caches_and_other_cpu
                0x400878b0                spi_flash_disable_interrupts_caches_and_other_cpu_no_os
                0x400878dc                spi_flash_enable_interrupts_caches_no_os
                0x400878f8                spi_flash_cache_enabled

My code does not use SPI at all, so this confuses me a bit.

I'm now lost......

The exception "Guru Meditation Error: Core 1 panic'ed (Interrupt wdt timeout on CPU1)" happens when the "loop()" takes too long to complete. If you have any "long running" loops inside "loop()" which prevents it from returning, you should make periodic calls to "yield()" within that/those loop/loops.

Lots 'a loops, hope you underknowledge! :wink:

Could be this was just a FastLED bug. I was consistently getting WDT timeouts. I updated to the latest FastLED version, and the exception seems to be cured. I do notice sometimes when I load a web page, the LEDs blink, which suggests an interrupting in the FastLED updates. I can live with that, though.

We'll see how it holds up with more testing...

Well, this is disappointing.... No more exceptions, but with this version of FastLED (v3.3.3), loading web pages causes the LEDs to glitch, with a random number of LEDs changing color as the page loads. Works fine as long as there are no web pages actively loading. This was no issue at all running on the 8266.

Not sure what to do now. I went back to the older FastLED (V3.3.2), and the crashes returned. With the latest version (v3.3.3) I get no crashes, but the LEDs flicker and blink during web page loads. A slight flicker during page loads I could maybe live with, but the problem is anywhere from a few to many LEDs change color, often several times, during a page load. Odd thing is, I am NOT refreshing them at all, so it appears something else is twiddling the LED bits?

what type of LEDs do you use? (I'm using APA102 as they don't put any time pressure on the system)

J-M-L:
what type of LEDs do you use? (I'm using APA102 as they don't put any time pressure on the system)

I'm using NeoPixels, the little tiny ones - forget the number.

I've got it mostly behaving, though the LEDs still glitch a bit. At least no more crashes. Going back to v3.3.2 of FastLED definitely makes it misbehave, so I suspect a bug somewhere in there.

An interesting comparison of the libraries for WS2812B here and explanation of how they are timing sensitive due to the protocol (and the more pixels you have the worse it gets).

You might want to try getting the current GitHub version or wait a few days for 3.4.0 (see this req from @ladyada and Mark Kriegsman’s answer). there has been some significant changes you would benefit from including this.

Are you using a logic level shifter or any other methods to increase the voltage on the data line? Have you considered moving the pixel updates to the second core to prevent the work (web-stuff) on the first core to have any impact on it?

EDIT: This seems to be a generic ESP32 + NeoPixels issue. Libraries which "bit-bangs" the data signal may cause a WDT timeout with long strips. Libraries using the built-in RMT module may suffer from artifacts when wifi is used. The latter may in some degree be solved by pinning the RMT task to core 1 and/or by putting the "strip.show()" call in a timer interrupt. Source of information. Also search for "esp32 neopixel flicker" and more info is available on the subject.

YEeah, it appears NeoPixels on ESP32 really don't work all that well, though they work perfectly when the WiFi is not too busy. I may play it safe and take another approach. I already have a small ATtiny1614 processor on the board that handles power management. I may move the NeoPixel function into that, and eliminate the problem altogether. It's got plenty of pins, RAM, FLASH and bandwdth to handle the job.

I know there's a "NeoPixel" library for ESP8266 that blasts the data out via DMA. Perhaps, there's a similar one for ESP32?

ESP32 Neopixel project code:
part 1

#include "sdkconfig.h"
#include "esp32/ulp.h"
#include "driver/rtc_io.h"
#include "esp_system.h" //This inclusion configures the peripherals in the ESP system.
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/timers.h"
#include "freertos/event_groups.h"
#include <Adafruit_NeoPixel.h>
#include "AudioAnalyzer.h"
////
/* define event group and event bits */
EventGroupHandle_t eg;
#define evtDo_AudioReadFreq       ( 1 << 0 ) // 1
////
TickType_t xTicksToWait0 = 0;
////
QueueHandle_t xQ_LED_Info;
////
const int NeoPixelPin = 26;
const int LED_COUNT = 24; //total number of leds in the strip
const int NOISE = 0; // noise that you want to chop off
const int SEG = 6; // how many parts you want to separate the led strip into
const int Priority4 = 4;
const int TaskStack40K = 40000;
const int TaskCore1  = 1;
const int TaskCore0 = 0;
const int AudioSampleSize = 6;
const int Brightness = 180;
const int A_D_ConversionBits = 4096; // arduino use 1024, ESP32 use 4096
const float LED0_Multiplier = 2.0;
////
Analyzer Audio = Analyzer( 5, 15, 36 );//Strobe pin ->15  RST pin ->2 Analog Pin ->36
// When we setup the NeoPixel library, we tell it how many pixels, and which pin to use to send signals.
Adafruit_NeoPixel leds = Adafruit_NeoPixel( LED_COUNT, NeoPixelPin, NEO_GRB + NEO_KHZ800 );
////
int FreqVal[7];//create an array to store the value of different freq
////
void ULP_BLINK_RUN(uint32_t us);
////
void setup()
{
  ULP_BLINK_RUN(100000);
  eg = xEventGroupCreate();
  Audio.Init(); // start the audio analyzer
  leds.begin(); // Call this to start up the LED strip.
  clearLEDs();  // This function, defined below, de-energizes all LEDs...
  leds.show();  // ...but the LEDs don't actually update until you call this.
  ////
  xQ_LED_Info = xQueueCreate ( 1, sizeof(FreqVal) );
  //////////////////////////////////////////////////////////////////////////////////////////////
  xTaskCreatePinnedToCore( fDo_AudioReadFreq, "fDo_ AudioReadFreq", TaskStack40K, NULL, Priority4, NULL, TaskCore1 ); //assigned to core
  xTaskCreatePinnedToCore( fDo_LEDs, "fDo_ LEDs", TaskStack40K, NULL, Priority4, NULL, TaskCore0 ); //assigned to core
  xEventGroupSetBits( eg, evtDo_AudioReadFreq );
} // setup()
////
void loop() {} // void loop
////
void fDo_LEDs( void *pvParameters )
{
  int iFreqVal[7];
  int j;
  leds.setBrightness( Brightness ); //  1 = min brightness (off), 255 = max brightness.
  for (;;)
  {
    if (xQueueReceive( xQ_LED_Info, &iFreqVal,  portMAX_DELAY) == pdTRUE)
    {
      j = 0;
      //assign different values for different parts of the led strip
      for (j = 0; j < LED_COUNT; j++)
      {
        if ( (0 <= j) && (j < (LED_COUNT / SEG)) )
        {
          set(j, iFreqVal[0]); // set the color of led
        }
        else if ( ((LED_COUNT / SEG) <= j) && (j < (LED_COUNT / SEG * 2)) )
        {
          set(j, iFreqVal[1]); //orginal code
        }
        else if ( ((LED_COUNT / SEG * 2) <= j) && (j < (LED_COUNT / SEG * 3)) )
        {
          set(j, iFreqVal[2]);
        }
        else if ( ((LED_COUNT / SEG * 3) <= j) && (j < (LED_COUNT / SEG * 4)) )
        {
          set(j, iFreqVal[3]);
        }
        else if ( ((LED_COUNT / SEG * 4) <= j) && (j < (LED_COUNT / SEG * 5)) )
        {
          set(j, iFreqVal[4]);
        }
        else
        {
          set(j, iFreqVal[5]);
        }
      }
      leds.show();
    }
    xEventGroupSetBits( eg, evtDo_AudioReadFreq );
  }
  vTaskDelete( NULL );
} // void fDo_ LEDs( void *pvParameters )
////
void fDo_AudioReadFreq( void *pvParameters )
{
  int64_t EndTime = esp_timer_get_time();
  int64_t StartTime = esp_timer_get_time(); //gets time in uSeconds like Arduino Micros
  for (;;)
  {
    xEventGroupWaitBits (eg, evtDo_AudioReadFreq, pdTRUE, pdTRUE, portMAX_DELAY);
    EndTime = esp_timer_get_time() - StartTime;
    // log_i( "TimeSpentOnTasks: %d", EndTime );
    Audio.ReadFreq(FreqVal);
    for (int i = 0; i < 7; i++)

part 2

    {
      FreqVal[i] = constrain( FreqVal[i], NOISE, A_D_ConversionBits );
      FreqVal[i] = map( FreqVal[i], NOISE, A_D_ConversionBits, 0, 255 );
      // log_i( "Freq %d Value: %d", i, FreqVal[i]);//used for debugging and Freq choosing
    }
    xQueueSend( xQ_LED_Info, ( void * ) &FreqVal, xTicksToWait0 );
    StartTime = esp_timer_get_time();
  }
  vTaskDelete( NULL );
} // fDo_ AudioReadFreq( void *pvParameters )
////
//the following function set the led color based on its position and freq value
//
void set(byte position, int value)
{
  // segment 0, red
  if ( (0 <= position) && (position < LED_COUNT / SEG) ) // segment 0 (bottom to top), red
  {
    if ( value == 0 )
    {
      leds.setPixelColor( position, 0, 0, 0 );
    } else {
      // increase light output of a low number
      // value += 10;
      // value = constrain( value, 0, 255 ); // keep raised value within limits
      if ( value <= 25 )
      {
        leds.setPixelColor( position, leds.Color( value , 0, 0) );
      } else {
        if ( (value * LED0_Multiplier) >= 255 )
        {
          leds.setPixelColor( position, leds.Color( 255 , 0, 0) );
        } else {
          leds.setPixelColor( position, leds.Color( (value * LED0_Multiplier) , 0, 0) );
        }
      }
    }
  }
  else if ( (LED_COUNT / SEG <= position) && (position < LED_COUNT / SEG * 2) ) // segment 1 yellow
  {
    if ( value == 0 )
    {
      leds.setPixelColor(position, leds.Color(0, 0, 0));
    }
    else
    {
      leds.setPixelColor(position, leds.Color( value, value, 0)); // works better to make yellow
    }
  }
  else if ( (LED_COUNT / SEG * 2 <= position) && (position < LED_COUNT / SEG * 3) ) // segment 2 pink
  {
    if ( value == 0 )
    {
      leds.setPixelColor(position, leds.Color(0, 0, 0));
    }
    else
    {
      leds.setPixelColor(position, leds.Color( value, 0, value * .91) ); // pink
    }
  }
  else if ( (LED_COUNT / SEG * 3 <= position) && (position < LED_COUNT / SEG * 4) ) // seg 3, green
  {
    if ( value == 0 )
    {
      leds.setPixelColor(position, leds.Color( 0, 0, 0));
    }
    else //
    {
      leds.setPixelColor( position, leds.Color( 0, value, 0) ); //
    }
  }
  else if ( (LED_COUNT / SEG * 4 <= position) && (position < LED_COUNT / SEG * 5) ) // segment 4, leds.color( R, G, B ), blue
  {
    if ( value == 0 )
    {
      leds.setPixelColor(position, leds.Color( 0, 0, 0));
    }
    else //
    {
      leds.setPixelColor(position, leds.Color( 0, 0, value) ); // blue
    }
  }
  else // segment 5
  {
    if ( value == 0 )
    {
      leds.setPixelColor(position, leds.Color( 0, 0, 0)); // only helps a little bit in turning the leds off
    }
    else
    {
      leds.setPixelColor( position, leds.Color( value, value * .3, 0) ); // orange
    }
  }
} // void set(byte position, int value)
////
void clearLEDs()
{
  for (int i = 0; i < LED_COUNT; i++)
  {
    leds.setPixelColor(i, 0);
  }
} // void clearLEDs()
//////////////////////////////////////////////
/*
  Each I_XXX preprocessor define translates into a single 32-bit instruction. So you can count instructions to learn which memory address are used and where the free mem space starts.

  To generate branch instructions, special M_ preprocessor defines are used. M_LABEL define can be used to define a branch target.
  Implementation note: these M_ preprocessor defines will be translated into two ulp_insn_t values: one is a token value which contains label number, and the other is the actual instruction.

*/
void ULP_BLINK_RUN(uint32_t us)
{
  size_t load_addr = 0;
  RTC_SLOW_MEM[12] = 0;
  ulp_set_wakeup_period(0, us);
  const ulp_insn_t  ulp_blink[] =
  {
    I_MOVI(R3, 12),                         // #12 -> R3
    I_LD(R0, R3, 0),                        // R0 = RTC_SLOW_MEM[R3(#12)]
    M_BL(1, 1),                             // GOTO M_LABEL(1) IF R0 < 1
    I_WR_REG(RTC_GPIO_OUT_REG, 26, 27, 1),  // RTC_GPIO2 = 1
    I_SUBI(R0, R0, 1),                      // R0 = R0 - 1, R0 = 1, R0 = 0
    I_ST(R0, R3, 0),                        // RTC_SLOW_MEM[R3(#12)] = R0
    M_BX(2),                                // GOTO M_LABEL(2)
    M_LABEL(1),                             // M_LABEL(1)
    I_WR_REG(RTC_GPIO_OUT_REG, 26, 27, 0),// RTC_GPIO2 = 0
    I_ADDI(R0, R0, 1),                    // R0 = R0 + 1, R0 = 0, R0 = 1
    I_ST(R0, R3, 0),                      // RTC_SLOW_MEM[R3(#12)] = R0
    M_LABEL(2),                             // M_LABEL(2)
    I_HALT()                                // HALT COPROCESSOR
  };
  const gpio_num_t led_gpios[] =
  {
    GPIO_NUM_2,
    // GPIO_NUM_0,
    // GPIO_NUM_4
  };
  for (size_t i = 0; i < sizeof(led_gpios) / sizeof(led_gpios[0]); ++i) {
    rtc_gpio_init(led_gpios[i]);
    rtc_gpio_set_direction(led_gpios[i], RTC_GPIO_MODE_OUTPUT_ONLY);
    rtc_gpio_set_level(led_gpios[i], 0);
  }
  size_t size = sizeof(ulp_blink) / sizeof(ulp_insn_t);
  ulp_process_macros_and_load( load_addr, ulp_blink, &size);
  ulp_run( load_addr );
} // void ULP_BLINK_RUN(uint32_t us)
//////////////////////////////////////////////

EXCVADDR: 0x00000000

First hint shows that a memory register or address is being set to 0 when it should not.

If there was a number, other than 0 then a array of some sort is being accessed incorrectly.

ESP32 uses SPI and SPI control signals for memory access.

BTW - My application is not driving a large number of NeoPixels - four strings: only 6, 9, 25, and 33 LEDs. The same code, running on ESP8266 had zero problems, except it sometimes ran out of memory on some web pages, so it has nothing to do with the loop time of my tasks. ESP32 completely solved the out of memory problem, but introduced the NeoPixel problem.

Idahowalker:
First hint shows that a memory register or address is being set to 0 when it should not.

If there was a number, other than 0 then a array of some sort is being accessed incorrectly.

ESP32 uses SPI and SPI control signals for memory access.

I suppose that explains why I kept seeing exception addresses within the SPI core code.

I’m not sure what the example code you posted shows? My problem is ENTIRELY when running NeoPixels WHILE loading web pages from the ESP32. The more content is on the pages, the more random anomalies I get on the NeoPixels, even when MY code is NOT updating them (I update them only on mode changes, which are rare). Between web page loads, the NeoPixels work 100% perfectly. Foruntately, so far, this appears to be only a minor cosmetic defect, everything else works exactly as it should, and the whole systems appears very stable. As I indicated, I think I will simply move the NeoPixel control to the existing ATtiny1614, and be done with it. I need to spin the PCB anyway, and it’s a simple change. It sure would be nice if FastLED worked better on ESP32. I never had a problem on the 8266…