ESP Critical Section Causes Crash

I have a simple task running on ESP32 that reads an I2C A/D, and updates an array with the data it reads, so other tasks can access the data. It all works perfectly, UNTIL I implement a critical section around the writes to the array, at which point it starts throwing exceptions.

Here is the function in question:

void readADC(void *parameter)
{
    for(;;)
    {
        switch(curADCChannel)
        {
            case 0:
                // Read mode ADC
                modeADCValue = analogRead(MODE_ADC_PIN);

                // Start conversion of first channel
                ADS1015.readADC_SingleEnded(curADCChannel, false);
                curADCChannel++;
                break;

            case 1:
            case 2:
            case 3:
            case 4:
                {
                    // Read result from last channel
                    int readchan = curADCChannel - 1;

                    portMUX_TYPE myMutex = portMUX_INITIALIZER_UNLOCKED;
                    taskENTER_CRITICAL(&myMutex);
                        ADCValues[readchan] = ADS1015.getLastConversionResults(false);
                    taskEXIT_CRITICAL(&myMutex);
    
                    if (curADCChannel < 4)
                    {
                        // Start conversion of next channel
                        ADS1015.readADC_SingleEnded(curADCChannel, false);
                        curADCChannel++;
                    }
                    else
                    {
                        curADCChannel = 0;
                    }
                }
                break;
        }
        vTaskDelay(2 / portTICK_PERIOD_MS);
    }
}

If I simply comment out the calls to taskENTER_CRITICAL and taskEXIT_CRITICAL, then it works perfectly.
The task is created by this:

    xTaskCreate(
        readADC,    // Function that should be called
        "readADC",  // Name of the task (for debugging)
        1000,       // Stack size (bytes)
        NULL,       // Parameter to pass
        1,          // Task priority
        NULL        // Task handle
    );

Here is the exception output:

Guru Meditation Error: Core  1 panic'ed (Interrupt wdt timeout on CPU1)
Core 1 register dump:
PC      : 0x40089f25  PS      : 0x00060534  A0      : 0x800888f0  A1      : 0x3ffba140
A2      : 0x3ffb7da0  A3      : 0x0000abab  A4      : 0xb33fffff  A5      : 0x00000001
A6      : 0x00060520  A7      : 0x0000cdcd  A8      : 0x0000cdcd  A9      : 0x3ffba140
A10     : 0x00000003  A11     : 0x00060523  A12     : 0x00060520  A13     : 0x00000001
A14     : 0x00060520  A15     : 0x00000000  SAR     : 0x00000017  EXCCAUSE: 0x00000006
EXCVADDR: 0x00000000  LBEG    : 0x00000000  LEND    : 0x00000000  LCOUNT  : 0x00000000

Backtrace: 0x40089f25:0x3ffba140 0x400888ed:0x3ffba170 0x40083705:0x3ffba1b0 0x40088f15:0x3ffba1d0

Core 0 register dump:
PC      : 0x4008b8d6  PS      : 0x00060c34  A0      : 0x8008aa93  A1      : 0x3ffb2510
A2      : 0x3ffb7d7c  A3      : 0x3ffb27e8  A4      : 0x00000001  A5      : 0x00000001
A6      : 0x00060c23  A7      : 0x00000000  A8      : 0x3ffb27e8  A9      : 0x3ffb27e8
A10     : 0x00000018  A11     : 0x00000018  A12     : 0x00000001  A13     : 0x00000001
A14     : 0x00060c23  A15     : 0x00000000  SAR     : 0x0000000a  EXCCAUSE: 0x00000006
EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000

Backtrace: 0x4008b8d6:0x3ffb2510 0x4008aa90:0x3ffb2530 0x40088d1b:0x3ffb2550 0x400e7eaa:0x3ffb2590 0x400e7f82:0x3ffb25b0 0x400e7716:0x3ffb25d0 0x400e3bf5:0x3ffb25f0 0x400e3eb2:0x3ffb2620 0x400e1bad:0x3ffb2640 0x400e1c62:0x3ffb2670 0x400e1cb1:0x3ffb26a0 0x400d2fc5:0x3ffb26c0 0x400d2ff8:0x3ffb26e0 0x400d1cf4:0x3ffb2700 0x40088f15:0x3ffb2730

Here is a decode of the exception:

PC: 0x40089f25: vTaskEnterCritical at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/portmux_impl.inc.h line 88
EXCVADDR: 0x00000000

Decoding stack results
0x40089f25: vTaskEnterCritical at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/portmux_impl.inc.h line 88
0x400888ed: xQueueGenericSend at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c line 734
0x40083705: ipc_task at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/ipc.c line 64
0x40088f15: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c line 143

What am I doing wrong?

Are you attempting to enclose an I2C transaction in a critical section? I’d assume that would not work unless I found evidence to the contrary.

I2C relies on interrupts to work correctly, when you enter a critical section all interrupts are disabled.

You don't need a critical section to write to a shared array. Use an array of type std::atomic (or use an std::mutex) to guard your shared data, not a critical section.

Pieter

I am curious why the task has such a low priority

And from: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/freertos-smp.html

Critical Sections & Disabling Interrupts: In ESP-IDF FreeRTOS, critical sections are implemented using mutexes. Entering critical sections involve taking a mutex, then disabling the scheduler and interrupts of the calling core. However the other core is left unaffected. If the other core attemps to take same mutex, it will spin until the calling core has released the mutex by exiting the critical section.

6v6gt:
Are you attempting to enclose an I2C transaction in a critical section? I’d assume that would not work unless I found evidence to the contrary.

Please look at the code I posted! The critical section surrounds ONLY a single write to a global variable. It has NOTHING to do with I2C. The I2C portion of the code works perfectly.

PieterP:
I2C relies on interrupts to work correctly, when you enter a critical section all interrupts are disabled.

You don't need a critical section to write to a shared array. Use an array of type std::atomic (or use an std::mutex) to guard your shared data, not a critical section.

Pieter

That is not the way it's done on the ESP32, which has two processors. The taskCRITICAL functions are specifically designed for the ESP32 multi-processor environment.

Idahowalker:
I am curious why the task has such a low priority

Because what it's doing is not terribly important? It should make no difference, since the code works perfectly WITHOUT any protection on the critical section, and the two processors are both lightly-loaded, so there is plenty of bandwidth to go around.

6v6gt:
And from: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/freertos-smp.html

Critical Sections & Disabling Interrupts: In ESP-IDF FreeRTOS, critical sections are implemented using mutexes. Entering critical sections involve taking a mutex, then disabling the scheduler and interrupts of the calling core. However the other core is left unaffected. If the other core attemps to take same mutex, it will spin until the calling core has released the mutex by exiting the critical section.

Which is exactly where I got the example that I used for implementing the critical section that crashes the program....

RayLivingston:
Please look at the code I posted! The critical section surrounds ONLY a single write to a global variable. It has NOTHING to do with I2C. The I2C portion of the code works perfectly.

Well, it wasn’t very clear from that code snippet you posted and you don’t appear to be very grateful for the responses you’ve received. Post a link to that ADS1015 library if you want someone to look at it for you.

RayLivingston:
Please look at the code I posted! The critical section surrounds ONLY a single write to a global variable. It has NOTHING to do with I2C. The I2C portion of the code works perfectly.

6v6gt is correct, and there's no reason for the hostile tone. Your code is doing I2C communication in the critical section, just have a look at the Adafruit_ADS1015::getLastConversionResults() function you're using.

RayLivingston:
That is not the way it's done on the ESP32, which has two processors. The taskCRITICAL functions are specifically designed for the ESP32 multi-processor environment.

I have plenty of experience with the ESP32 and thread-safe programming. Atomic variables and mutexes are also specifically for multi-processor environments.
taskENTER_CRITICAL disables interrupts and stops the scheduler, which is totally overkill for protecting shared data, it uses locks, disables interrupts, and stops scheduling, all of those are unnecessary. You're just writing a single integer, you don't need a lock, just an std::atomic, you're not accessing the shared data inside of an ISR, so disabling interrupts is pointless, and there's nothing to be gained from disabling the scheduler either.

Just use atomic variables to share the data between tasks.

PieterP:
taskENTER_CRITICAL disables interrupts and stops the scheduler, which is totally overkill for protecting shared data, it uses locks, disables interrupts, and stops scheduling....

My reading of the docs says that's true. But, it can only disable interrupts and stop scheduling on the Core that calls taskENTER_CRITICAL(). In order for the scheme to work, the OTHER Core must play by the rules. Meaning it can't try to access the resources being protected by the critical section unless it obtains the lock (mutex). To do so, it must also call taskENTER_CRITICAL() using the same portMUX_TYPE * variable. This is different than a critical section in a single-core environment. There you just need to disable interrupts and stop scheduling. That's enough to assure you that that the protected resources will only be accessed by the code/task in the critical section.

My Reasoning:

Critical Sections & Disabling Interrupts: In ESP-IDF FreeRTOS, critical sections are implemented using mutexes. Entering critical sections involve taking a mutex, then disabling the scheduler and interrupts of the calling core. However the other core is left unaffected. If the other core attemps to take same mutex, it will spin until the calling core has released the mutex by exiting the critical section.

From the library link supplied by @peterp, it also appears that methods has a delay() statement in it which I guess relies on a timer interrupt.

6v6gt:
From the library link supplied by @peterp, it also appears that methods has a delay() statement in it which I guess relies on a timer interrupt.

Indeed. The FreeRTOS tick interrupt.

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.