ArduinoCore-Zephyr (including Q) 0.53.0 was released today

Looks like it was released this morning, and both Arduino IDE 2.x and App Lab
has been updated :smiley:

Lots of good things have been fixed, like for example I can now use kernel events with threads. Example scrolling text on the matrix using a thread and events:

#include "ArduinoGraphics.h"
#include "Arduino_LED_Matrix.h"

Arduino_LED_Matrix matrix;

struct k_event my_event;
volatile bool thread_active = false;
volatile uint32_t loop_count = 0;
#define STACK_SIZE 1024
#define PRIORITY 7

K_THREAD_STACK_DEFINE(thread1_stack, STACK_SIZE);
struct k_thread thread1_data;
k_tid_t thread1_tid;
char led_text[20];

void thread1(void *p1, void *p2, void *p3) {
  UNUSED(p1);
  UNUSED(p2);
  UNUSED(p3);
  uint32_t events;
  uint32_t thread_loop_count = 0;
  while (1) {
    events = k_event_wait(&my_event, 0xFFF, true, K_MSEC(2000));
    if (events == 0) {
      sprintf(led_text, "Timeout");
    } else {
      thread_loop_count++;
      sprintf(led_text, "L:%u t:%u", loop_count, thread_loop_count);
    }
    matrix.beginText(0, 0, 127, 0, 0);  // X, Y, then R, G, B
    matrix.print(led_text);
    matrix.endText(SCROLL_LEFT);
    thread_active = false;
  }
}


void setup() {
  // put your setup code here, to run once:
  pinMode(LED_BUILTIN, OUTPUT);
  matrix.begin();
  matrix.textFont(Font_5x7);
  matrix.textScrollSpeed(100);
  matrix.clear();
  k_event_init(&my_event);
  thread1_tid = k_thread_create(&thread1_data, thread1_stack, K_THREAD_STACK_SIZEOF(thread1_stack),
                                thread1, NULL, NULL, NULL, PRIORITY, 0, K_NO_WAIT);
}

void loop() {
  loop_count++;
  digitalWrite(LED_BUILTIN, (loop_count & 1) ? HIGH : LOW);

  if (!thread_active) {
    thread_active = true;
    k_event_set(&my_event, 0x001);
  }
  delay(250);
}

There may have been some improvements on how much memory you can malloc at run time over the previous release. Where on some boards now we can at least maybe be able to allocate a frame buffer for smaller displays or cameras. Like an ILI9341 (320x240x2) = 153600
However probably not for slightly larger display like ILI9488 or ST7796 (480x320x2) = 307200

Simple sketch:

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  while (!Serial && millis() < 5000) {}
  delay(250);
  Serial.println("Check max malloc");
  Serial.print("Enter max KB to start: ");
  pinMode(LED_BUILTIN, OUTPUT);

  //printk("KB: %u Buffer: %p\n", buffer);
}

void loop() {
  digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));
  if (Serial.available()) {
    int kb_max = Serial.parseInt();
    int kb;
    void *buffer;
    Serial.println(kb_max);
    for (kb = kb_max; kb > 0; kb--) {
      buffer = malloc(kb * 1024);
      if (buffer) break;
    }
    Serial.print(kb);
    Serial.print(": ");
    Serial.print((uint32_t)buffer, HEX);
    if (buffer) {
      Serial.print(" - ");
      Serial.print((uint32_t)buffer + kb * 1024, HEX);
    }
    Serial.print(" Stack var: ");
    Serial.println((uint32_t)&buffer, HEX);
    free(buffer);    
    Serial.print("Enter max KB to start: ");
    while (Serial.read() != -1) {}
  }
  delay(500);
}

I tried this on a few of the boards, only the Q has enough for these larger ones:

Q: 566K
Portenta H7: 270
GIGA: 233
Portenta C33: 25
Nicla Sense: 8

With this now released, I am wondering if this is a good time to go through the Github project and do something similar like Zephyr does, and clear out any issues/pr's we issued that have not been commented on or addressed over some period of time and/or releases?

Please only do that if one of the following is true:

  • You verify that the bug/enhancement that is the subject of an issue has already been resolved.
  • You verify that the change proposed by the PR has been superseded by other development work in the project.
  • You verify that the issue or PR is a duplicate of another issue/PR, and the other item is either higher quality, or else equal quality and submitted earlier. In the alternate case where the other issue/PR is lower quality or submitted later, you can comment on that issue to notify the maintainers of the existence of the duplicate so we can close it.

It is definitely helpful to close items that match one of those criteria.

If you close an item, please take the time to add a comment explaining the reason for closure.

Please don't close items just because you didn't get a reply. We rely on the issues and PRs to track these tasks. If you close them, that pulls the rug out from under our maintenance efforts. I understand that it is frustrating when you don't see prompt action on an issue or pull request. Trust me that we are also disappointed that the finite resources available are not sufficient to allow us to maintain the projects in an optimal manner. But arbitrarily closing issues and pull requests only makes things more difficult because then we must figure out why the item was closed and whether we need to create a brand new issue/PR in order to be able to continue to track the task. There are many notifications of such closures in my email inbox and I simply can't find the time to investigate and act on them.

We close any issue or PR that we have determined to be invalid, infeasible, or not aligned with our vision for the project. If the issue or PR is still open, it means we want it to still be open (except in the cases mentioned above where it is objectively resolved or duplicate and we just didn't noticed that it should have been closed). Note that a lot of the project management work is done via private communication channels (e.g., our internal Kanban boards and Slack). So it might well be that there are plans in progress regarding an issue or PR even though you don't see any indication on GitHub.

@KurtE

Thats good news but unfortunately things like mutexes and semaphores are still throwing errors - only tested on the Q but know how the core works going to be the same on other boards as well.

DYNAMIC:

*** Booting Zephyr OS build v4.2.0-38-g994b835f5936 ***
[00:00:00.205,000] <err> llext: Region 1 ELF file range (0x19a8-0x1bcf) overlaps with 2 (0x1a08-0x1bc7)
[00:00:00.215,000] <err> llext: Failed to map ELF sections, ret -8
Failed to load sketch, rc -8

Static

*** Booting Zephyr OS build v4.2.0-38-g994b835f5936 ***
System Heap end: 0x20031e30
System Heap start: 0x20029e30
▒Tmain TID:20000558 pri:50dd0, size 0x20000
hreading time slice sketch  started
main TID:Resource given and available_instance_count = 11
[00:00:00.224,000] <err> os: ***** BUS FAULT *****
[00:00:00.230,000] <err> os:   Precise data bus error
[00:00:00.236,000] <err> os:   BFAR Address: 0x0
[00:00:00.241,000] <err> os: r0/a1:  0x200010f0  r1/a2:  0x280f18ed  r2/a3:  0x00000000
[00:00:00.250,000] <err> os: r3/a4:  0x080f0d98 r12/ip:  0x00000000 r14/lr:  0x08016f59
[00:00:00.258,000] <err> os:  xpsr:  0x01000200
[00:00:00.263,000] <err> os: s[ 0]:  0x080f03a9  s[ 1]:  0x20000570  s[ 2]:  0x00000010  s[ 3]:  0x08016f59
[00:00:00.274,000] <err> os: s[ 4]:  0x00000000  s[ 5]:  0x200010d8  s[ 6]:  0x00000000  s[ 7]:  0x0800ded5
[00:00:00.284,000] <err> os: s[ 8]:  0x080f03a9  s[ 9]:  0x080079ad  s[10]:  0x080f0435  s[11]:  0x080f0477
[00:00:00.295,000] <err> os: s[12]:  0x00000000  s[13]:  0x00000000  s[14]:  0x00000000  s[15]:  0x08005f17
[00:00:00.305,000] <err> os: fpscr:  0x00000000
[00:00:00.310,000] <err> os: Faulting instruction address (r15/pc): 0x0800f182
[00:00:00.318,000] <err> os: >>> ZEPHYR FATAL ERROR 25: Unknown error on CPU 0
[00:00:00.326,000] <err> os: Current thread: 0x20000ef8 (producer_id)
[00:00:00.333,000] <err> os: Halting system

Here is the test sketch

#include <zephyr/random/random.h>
//#include "LibPrintf.h"
#include "elapsedMillis.h"
elapsedMillis timer;

#define PRODUCER_STACKSIZE 1024
#define CONSUMER_STACKSIZE 1024

/* STEP 2 - Set the priority of the producer and consumper thread */
#define PRODUCER_PRIORITY 5
#define CONSUMER_PRIORITY 5

/* STEP 9 - Define semaphore to monitor instances of available resource */
K_SEM_DEFINE(instance_monitor_sem, 10, 10);

/* STEP 3 - Initialize the available instances of this resource */
volatile uint32_t available_instance_count = 10;

// Function for getting access of resource
void get_access(void)
{
	/* STEP 10.1 - Get semaphore before access to the resource */
	k_sem_take(&instance_monitor_sem, K_FOREVER);

	/* STEP 6.1 - Decrement available resource */
	available_instance_count--;
	printk("Resource taken and available_instance_count = %d\n", available_instance_count);
}

// Function for releasing access of resource
void release_access(void)
{
	/* STEP 6.2 - Increment available resource */
	available_instance_count++;
	printk("Resource given and available_instance_count = %d\n", available_instance_count);

	/* STEP 10.2 - Give semaphore after finishing access to resource */
	k_sem_give(&instance_monitor_sem);
}

/* STEP 4 - Producer thread relinquishing access to instance */
void producer(void)
{
	Serial.print("Producer thread started\n");
	while (1) {
		release_access();
		// Assume the resource instance access is released at this point
		k_msleep(500 + sys_rand32_get() % 10);
	}
}

/* STEP 5 - Consumer thread obtaining access to instance */
void consumer(void)
{
	Serial.print("Consumer thread started\n");
	while (1) {
		get_access();
		// Assume the resource instance access is released at this point
		k_msleep(sys_rand32_get() % 10);
	}
}

K_THREAD_DEFINE(producer_id, PRODUCER_STACKSIZE, producer, NULL, NULL, NULL, PRODUCER_PRIORITY, 0,
		0);
K_THREAD_DEFINE(consumer_id, CONSUMER_STACKSIZE, consumer, NULL, NULL, NULL, CONSUMER_PRIORITY, 0,
		0);

void setup() {
  Serial.begin(115200);
  while (!Serial && millis() < 5000) {};
  Serial.println("Threading time slice sketch  started");
k_tid_t our_tid = k_sched_current_thread_query();
  int main_pri = k_thread_priority_get(our_tid);
  Serial.print("main TID: ");
  Serial.print((uint32_t)our_tid, HEX);
  Serial.print(" pri: ");
  Serial.println(main_pri);
  printk("main TID:%x pri:%d\n", (uint32_t)our_tid, main_pri);
  //k_thread_priority_set(our_tid, THREAD0_PRIORITY+1);
  //main_pri = k_thread_priority_get(our_tid);
  //Serial.print("\tupdated pri: ");
  //Serial.println(main_pri);
  //printk("main TID:%x pri:%d\n", (uint32_t)our_tid, main_pri);
}

void loop() {
  if(timer > 5000){
    Serial.println("called from loop");
    timer = 0;
  }
  k_msleep(10);
}

Updated: Simple Thread example fails with region overlap · Issue #243 · arduino/ArduinoCore-zephyr

I just checked, and Arduino IDE is still 2.3.7.

That is very disappointing.
For a multi-tasking OS like Zephyr those are required elements. It should never have got to GA without those features working.

The release @KurtE reported here is of the boards platform, not of Arduino IDE. Boards platform updates are performed via the Arduino IDE Boards Manager. This system allows Arduino (and 3rd parties as well) to distribute new releases of platforms without also having to make a new release of the IDE.

The arduino/ArduinoCore-zephyr codebase is distributed as multiple separate boards platforms. The most significant of these is the "Arduino UNO Q Board" platform (machine identifier arduino:zephyr) that adds support for the UNO Q to Arduino IDE and Arduino App Lab.

If you are using Arduino IDE, you can check in Boards Manager to see which version of the platform you have installed. The platform version is not displayed in Arduino App Lab, but it will automatically show a dialog that offers to update to the new version of the platform.

I can confirm both updates in the IDE and using Arduino App Lab.

==> RTOS

I did not see any reference to Boards, but maybe I should have known that by the absence of a new 2.x release ver for the IDE.

Isn't Zephyr the RTOS replacement for some boards?

As I know Zephyr is the name of a RTOS developed by Linux Foundation installed in UNO Q like the FreeRTOS developed by Real Time Engineers Ltd. installed in ESP32.

done..

i also see we got a cli update as well..

been busy on the linux side i haven’t dived down into zephyr all that much except to say it’s very different from the esp32 FreeRTOS..

most interesting is we don’t compile everything but rather just our apps to be run by the OS..

faster compiles and uploads..

much to learn..

~q

That may be, but the new OS is called Zephyr, RTOS is the old.
According to Arduino docs.

running Arduino sketches over Zephyr OS

Here is a blog arduino published a while ago

The Q is simply another board that is built on Zephyr. Today’s release actually caused 3 possible board installs to be updated.

Q, Arduino zephyr, contributed boards

that’s depressing, no synchronization primitives..

why have threads at all..

~q

YEP, I have some SEEED boards that mention Zephyr.

Some of them are working. I believe the main issue is with:

Which I will probably not work work within an LLEXT which is where are Arduino
code resides.

Here is a code that uses mutex, which does run:

Right now running it on Portenta H7.
K_MUTEX_DEFINE(my_mutex);

void thread1(void *p1, void *p2, void *p3)
{
    while (1) {
        k_mutex_lock(&my_mutex, K_FOREVER);
        printk("Thread 1 is accessing the shared resource\n");
        k_sleep(K_MSEC(1000));
        k_mutex_unlock(&my_mutex);
        k_sleep(K_MSEC(1000));
    }
}

void thread2(void *p1, void *p2, void *p3)
{
    while (1) {
        k_mutex_lock(&my_mutex, K_FOREVER);
        printk("Thread 2 is accessing the shared resource\n");
        k_sleep(K_MSEC(1000));
        k_mutex_unlock(&my_mutex);
        k_sleep(K_MSEC(1000));
    }
}

#define STACK_SIZE 1024
#define PRIORITY 7

K_THREAD_STACK_DEFINE(thread1_stack, STACK_SIZE);
struct k_thread thread1_data;
k_tid_t thread1_tid;

K_THREAD_STACK_DEFINE(thread2_stack, STACK_SIZE);
struct k_thread thread2_data;
k_tid_t thread2_tid;

void setup() {
  thread1_tid = k_thread_create(&thread1_data, thread1_stack, K_THREAD_STACK_SIZEOF(thread1_stack),
    thread1, NULL, NULL, NULL, PRIORITY, 0, K_NO_WAIT);

  thread2_tid = k_thread_create(&thread2_data, thread2_stack, K_THREAD_STACK_SIZEOF(thread2_stack),
    thread2, NULL, NULL, NULL, PRIORITY, 0, K_NO_WAIT);


}

void loop() {
  // put your main code here, to run repeatedly:
  k_msleep(1);
}

They do have primitives. The issue is that there are a few different pieces
involved:
a) Zephyr - zephyrproject-rtos/zephyr: Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.

Kernel Services — Zephyr Project Documentation

b) ArduinoCore-Zephyr arduino/ArduinoCore-zephyr: Arduino Core based on Zephyr+llext

With these releases (or if you build it yourself) - The first thing that happens (After you synced up and the like) is to first build what Arduino is calling the
bootloader. It is actually the Zephyr build of the system, with all of the device
tree and kernel code and device drivers and the like. All of these things are statically defined. That is what the "burn the bootloader" is doing, is it is
installing this code onto your board.

Part of this is the llext which is a loader of extensions, and this is what your builds of the Arduino code is producing.

The issue here is that if your code wishes to use things provided within the
main zephyr code, there has to be some way to resolve where that code is
within your current "bootloader". Such as where is k_mutex_lock located?

As part of the build, there is more or less a jump table that is used that
says k_mutex_lock is at address Y and it calls to that location. Well suppose
k_mutex functions were not included in the table, then the load fails.

Or if the one who decided which features of Zephyr to include with the current
build does not enable events...

That is what was happening before with events. And that was
fixed in the PR:

arduino/ArduinoCore-zephyr: Arduino Core based on Zephyr+llext

Which this is the first release with this merged in

@KurtE
Yep you are write - forgot that they are not supporting dynamic threads but only static. Here is descent link that explains the difference for those interested
Two Ways to Create Threads in Zephyr RTOS

Yeah, it is a bit confusing. It was mentioned previously, but I'll provide a more detailed description in case it will be helpful:

The Arduino developers have created an Arduino Arduino core based on Zephyr OS. That core, and its associated assets, is hosted in a GitHub repository named "ArduinoCore-zephyr":

Since much of the architecture-specific code is in the Zephyr codebase, it is possible for support for a diverse variety of target boards to be added to the ArduinoCore-zephyr project. For example, it supports the Nordic nRF52840-based Nano 33 BLE boards as well as the STMicroelectronics STM32U585-based UNO Q. The number of supported boards will continue to grow as time goes on. We learned from experience that distributing the support for such a diverse array of boards via a single Arduino boards platform results in the platform becoming excessively bloated over time, which has harmful effects on usability. For this reason, the approach is taken of instead using multiple platforms, each of which provides support for a smaller set of targets.

This results in a confusing situation where there isn't a 1:1 correlation between the codebase and a single distributed Arduino boards platform (as has traditionally been the case with projects such as the "Arduino AVR Boards" platform). So when we are referring to the codebase and its development repository, we will say something like "ArduinoCore-zephyr" or arduino/ArduinoCore-zephyr. Conversely, when we are referring to the platform we will say something like "Arduino UNO Q Board platform", or perhaps use the machine identifier like arduino:zephyr. To make matters more confusing, there is a big problem of the community and even the Arduino company being very inconsistent in how they refer to these things. So you might see someone write something like "Zephyr core" when the correct term would be "Arduino UNO Q Board platform".

Earlier today, the developers made an 0.53.0 release in the arduino/ArduinoCore-zephyr repository:

They then made a release with the same version number of each of the Arduino boards platforms that are derived from that codebase:

  • "Arduino UNO Q Board": Provides support for the UNO Q.
  • "Arduino Zephyr Boards (BETA)": Provides support for official boards other than the UNO Q.
  • "Zephyr Community Boards (BETA)": Provides support for 3rd party boards.

So from a developer perspective, the most significant event was a release of the "ArduinoCore-zephyr" project. From a user perspective, the significant event is that Arduino IDE or Arduino App Lab is offering an update of the "Arduino UNO Q Board" platform.

RTOS is an acronym for "real-time operating system", which is the term for a general type of operating system:

There is no specific software project named "RTOS".

As is mentioned by the blog post shared by @KurtE, Arduino had previously adopted the Mbed OS operating system upon which to build an Arduino core. So when referring to the official Arduino boards platforms, it would be most accurate to say:

the new OS is called Zephyr, Mbed OS is the old.

However, other operating systems are also in use in the Arduino ecosystem. You may be thinking of the popular FreeRTOS:

Thanks for the clarification. I was aware that there were several packages, but did not really understand why. I just looked and I was missing the Community set.

I know it's FreeRTOS, I just mis-spoke. I did not know about Mbed OS.