Question about multiple concurrent tasks using Serial, and Wire

Working with an ESP32-S3, specifically the Adafruit Qualia ESP32-S3 for RGB666 Displays.

I'm working on an application where I needed to move some of the program code into a task on the other core (due to needed delay() calls). It's working mostly, but I have occasional hangs/crashes.

The main task (using loop, on core 1) reads from an SD card (connected using SPI) and displays frames of an MJPEG to a screen. It also reads inputs from 2 I2C sensor boards, and sometimes sends an output to an I2C GPIO extender.

I have a task that I created at the end of setup that is pinned to core 0 (I'm not using the wifi or btle) that is responsible for communicating with an I2C motor controller, and occasionally triggering an ouput on an I2C GPIO extender.

There are 2 shared global state variables that are shared between the 2 tasks. One of these is a trigger from the main task to tell the other to start doing it's thing (moving a motor). When done, it sets this variable back to 0. The other shared variable is one mainly used by the main loop to indicate when it triggers the GPIO extender.

Also shared are the global object for the GPIO extender (since both tasks have to send it messages) and Serial, since I use serial for debug output while testing the app.

I have mutexes around the places where I am setting shared variables (I assume a read doesn't need to be locked) but not around things like calls to Serial or when using I2C library functions (I2C GPIO extender, I2C motor controller, etc.) Do calls to all of these need to be wrapped in taskENTER_CRITICAL / taskEXIT_CRITICAL or should I look elsewhere for the cause of the random crashes?

Welcome!
Sounds like you might be talking about an ESP32 - how about telling us for sure? While you're thinking about that, please read:

which I have to assume you blew past in haste in order to post your question.

Thanks. I read it, and expanded on my original question with the specific board, and some additional background on what I am trying to do. (sorry about that, you're right, I have been messing with this all last night and this morning, and was kind of rushing)

Speaking of the crashes, is there a way to get the cause, or a stack trace, or something when it does happen? What I am seeing is the Serial output just stops at a println call, and my PC opens the bootloader drive on my PC. I googled and people seem to indicate that Serial should print out a stacktrace, but I'm not getting any. (Using the USB based serial, not any dedicated serial output pin, I don't think this particular board exposes a separate pin)

A few points:

  1. Delays are never “needed”. Even if programming in the classic setup() / loop() Arduino paradigm, delays can be totally avoided through proper non-blocking code techniques.

  2. There’s no reason to use two cores for the functions you describe. They can be handled on a single core with proper interaction and coordination between the tasks.

  3. As long as the shared data structures are not accessed in ISR context, Critical Sections are not needed. Mutexes will be sufficient.

  4. Mutexes are needed for both reading and writing the data structures unless such accesses are atomic.

  5. You handwaving explanation of the code’s structure and problems with it is insufficient to provide further assistance. For more detailed help, post a complete code along with description of the “crashes”.

  6. ESP32s always output stack traces for things like stack overruns, watchdog timer activations, illegal data / code accesses, etc. I don’t know which port is used for this on ESP32-S3, but I suspect it will be a UART-based one. You can test by causing an intentional WDT activation and seeing where the trace appears.

Technically, yes, needed is a strong word, and there are ways to work around it, but it makes the overall code (which is already going to be large and very complex) much harder to follow and trace through. Most times I opt for readability and easier understanding of the code instead of just doing cool things. Especially as others will be using it as well, not just me.

The variables are all read/write atomic. I'm not doing any sort of read-then-update work with any of them, that's why I didn't have any sort of mutex around them, and I was unsure if they would cause the system to crash (I know C on Unix, it wouldn't, but I'm new to arduino) I did remove the calls to Serial.print() in the other thread, and it's not crashing any longer, so I'm guessing I would need protections around any Serial output, though.

If you want the full code, I can send you a link to the Github repo. it's only about 1000 lines, so not too big (yet)

As for the nature of the crash, all i know is the screen goes blank, Serial output just stops, and Windows pops up a notification about the bootloader drive being available (i.e. where you would upload a new uf2 file, as if I had double-clicked reset button) I will check to see if there is another serial output, but so far I've only been using the USB presented one.

I am thinking of separating this out further into 3 simultaneous tasks: One for the main loop (read inputs, set overall state, send notifications to the other 2 tasks), One for the video player loop (get the file to play in a notification, and loop through it repeatedly until another notification for file change is received), and finally one task for the other outputs (The 2 GPIO extenders for external triggers, and the motor controller changes.)

Without debugging the code this is more a general discussion about what may be going wrong. Reading your post I thought about the following:

  1. Did you reserve enough stack for your tasks? You can check stack usage high water marks by calling uxTaskGetStackHighWaterMark (NULL) and always add some additional memory to it.

  2. Did you declare global variables volatile? Normally omitting volatile keyword would not cause any problem, but perhaps it is worth checking.

  3. Hangs could also be caused by deadlocks. If you are using many mutexes they should always be accessed in the same order to avoid deadlocks. Perhaps using only one mutex would be enough?

In general it is better to let ESP32 handle the cores itself, at least according to my experience.

Maybe you missed this part when viewing "How to get the best out of this forum"

Sure, please post the link, I'd like to see what you are doing. But honestly, I'm not particularly keen on debugging other programmers' code. And multitasking bugs are particularly difficult to find.

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.