Is there a list of "core" libraries that are "built-into" app lab apps?

I notice in AppLab, that while I do have to #include <Arduino_LED_Matrix.h>, I do not have to manually import the library.

However, with Arduino_RouterBridge , I have to manually import it AND include it.

Is there a list of libraries somewhere that are “built-in” and I do not have to manually add them?

Better yet, are there plans to have some libraries become part of the “core app structure”, so I don’t have to include them OR import them?

I feel like Arduino_RouterBridge would be the perfect candidate for this!

Hi @programming_electronics_academy.

This is incorrect. You are probably thinking of the "MsgPack" library.

The "Arduino_RouterBridge" library is currently bundled with the "Arduino UNO Q Board" platform, so it is not necessary to "install" it in order to use it in your Apps or standalone sketches. It may be removed from the platform at some point in the future. If so, it would indeed need to be "installed" in every App you want to use it with after updating to the future version of the "Arduino UNO Q Board" platform that has such a change.

Here they are:

The correct term is "platform bundled" library.

The only relevant thing I am aware of is this community request:

When using Arduino IDE all libraries we have installed are accessible to any sketch. When using Arduino Cloud Editor, we usually don't even need to install libraries because all the thousands of libraries in the Arduino Library Manager are preinstalled on the Arduino Cloud server. For those of us accustomed to using these tools, the need to configure the set of sketch library dependencies for each App might seem inconvenient. However, there is a very good reason why Arduino App Lab was intentionally designed this way. The problem with the old system where all your projects share the same dependencies environment is that it makes your projects very fragile. Updating or installing a library for one project might break your other projects.

Each Arduino App uses a dedicated isolated dependencies environment. Beyond the "Arduino UNO Q Board" platform, that environment only contains the specific library dependencies you have "installed" for that App. This is a huge advancement over the old system because it means that you don't have to worry about your project being broken when you come back to it at some point in the future due to changes in the libraries you have installed.

Some of the old timers here use a separate dedicated "portable" installation of Arduino IDE for each of their projects just to achieve an isolated dependencies environment.

Thanks for this @ptillisch - makes a bunch of sense, and I appreciate knowing what to call things :slight_smile:

OK, maybe I am not using the right word, but when I say “Install”, I guess I mean “Add?” by using the little + sign next to the Sketch Libraries folder.

In this AppLab example App:

Here, the Sketch Libraries folder has Arduino_RouterBridge in it, and the #include line in the sketch.ino file.

But Arduino_LED_Matrix only has the #include line and is not in the Sketch libraries folder.

I am trying to understand why I have to “add” the Arduino_RouterBridge library, but Arduino_LED_Matrix library does not need added.

"Install" is right. "Install" is the standard term, as defined by the designers of the Arduino App Lab user interface:

The reason I added quotes to the term in my previous reply is to indicate that I was using the term as it applies to Arduino Apps, as opposed to the general meaning of the term.

Unlike the Arduino IDE or Arduino Cloud Editor Library Managers, the "Install" button in the "Add sketch library" dialog of Arduino App Lab does not actually perform an installation in the traditional sense of transferring software to the file system. What it does instead is to add an entry for that library to the build profile in the sketch/sketch.yaml sketch project file. This build profile defines the dependencies that will be present in the isolated environment in which the sketch is compiled.

The dependencies environment of the sketch is generated or updated as needed when it is compiled (as triggered by running the App).

When you say "Sketch Libraries folder". Are you referring to the .cache/sketch/libraries subfolder of the App?

As I explained, this library is bundled with the "Arduino UNO Q Board" platform. You can see the library under the following path on your UNO Q's drive;

~/.arduino15/packages/arduino/hardware/zephyr/0.53.1/libraries/

And I'm trying to explain to you that, just as with the Arduino_LED_Matrix library, you do not need to "install" the Arduino_RouterBridge library.

Please perform this experiment:

  1. Start Arduino App Lab (and connect it to your UNO Q board if using in PC hosted mode).
  2. Select "My Apps" from the bar on the left hand side of the Arduino App Lab window.
  3. Click the "Create new app +" button.
    A menu will open.
  4. Select "Create New App" from the menu.
    The "Create new app" dialog will open.
  5. Enter any name in the "Insert title" field of the dialog.
  6. Click the "Create new" button.
    A new App will be created.
  7. Select sketch > sketch.ino from the "Files" panel at the left hand side of the Arduino App Lab window.
    The sketch.ino file will open in the editor panel of the Arduino App Lab window.
  8. Add the following line to the top of the sketch.ino file:
    #include <Arduino_RouterBridge.h>
    
  9. Click the "▸ Run" button at the top right corner of the Arduino App Lab window.
  10. Watch the output that is printed in the "App launch" panel.
    The App fails to launch, with the following compiler error shown in the "App launch" panel:
    In file included from /home/arduino/.arduino15/packages/arduino/hardware/zephyr/0.53.1/libraries/Arduino_RPClite/src/client.h:14,
    from /home/arduino/.arduino15/packages/arduino/hardware/zephyr/0.53.1/libraries/Arduino_RPClite/src/Arduino_RPClite.h:22,
    from /home/arduino/.arduino15/packages/arduino/hardware/zephyr/0.53.1/libraries/Arduino_RouterBridge/src/bridge.h:30,
    from /home/arduino/.arduino15/packages/arduino/hardware/zephyr/0.53.1/libraries/Arduino_RouterBridge/src/Arduino_RouterBridge.h:16,
    from /home/arduino/ArduinoApps/1433789-2/sketch/sketch.ino:1:
    /home/arduino/.arduino15/packages/arduino/hardware/zephyr/0.53.1/libraries/Arduino_RPClite/src/error.h:20:10: fatal error: MsgPack.h: No such file or directory
    20 | #include "MsgPack.h"
    |          ^~~~~~~~~~~
    compilation terminated.
    exit status 1
    
  11. Select the "Editor" tab at the top of the Arduino App Lab window.
  12. Click the button that looks like a book overlayed with a + symbol in the "Sketch Libraries" panel on the left hand side of the Arduino App Lab window.
    The "Add sketch library" dialog will open.
  13. Type MsgPack in the "Search library" field of the dialog.
  14. Click the "Install" button in the entry for the "MsgPack" library in the search results.
  15. Click the X icon at the top right corner of the "Add sketch library" dialog.
    The dialog will close.
  16. Click the "▸ Run" button at the top right corner of the Arduino App Lab window.

This time, the App will launch successfully. Do you see now that you do not need to "install" the Arduino_RouterBridge library?

Thanks for this @ptillisch ! I am normally pretty slow to pick things up and I really appreciate you walking the dog with me on this :slight_smile:

OK - I think I see my disconnect now.

I went through your numbered example with success. I installed the MsgPack library by hideakitai (V0.4.2), and it worked like you said.

I am referring to this left side section in the App Lab - it says “Add Sketch Library”

Here is where my disconnect still is…

When I search for and install the Arduino_RouterBridge by Arduino (V0.3.0), **my bridge code runs fine.
**
Looking at the sketch.yaml file, after installing just Arduino_RouterBridge, it looks like this:

profiles:
default:
fqbn:
platforms:
- platform: arduino:zephyr
libraries:
- dependency: Arduino_RPClite (0.2.1)
- Arduino_RouterBridge (0.3.0)
- dependency: ArxContainer (0.7.0)
- dependency: ArxTypeTraits (0.3.2)
- dependency: DebugLog (0.8.4)
- dependency: MsgPack (0.4.2)

default_profile: default

So MsgPack is getting added as a dependency - not sure where/how it gets added (I don’t see it as an include in the Arduino_RouterBridge source).

Also, if I remove Arduino_RouterBridge, and instead install the MsgPack library, my bridge code runs fine.

As I was playing around with AppLap example Apps that use bridge code, I removed the Arduino_RouterBridge library and the App stopped working, thus I concluded having Arduino_RouterBridge installed was a requirement.

But now (after your reply and closer inspection of the console and the yaml file ) I see that MsgPack is actually what it required, however, it appears that by installing the Arduino_RouterBridge library, it adds MsgPack as a dependency for me.

As an aside: It is worth noting, that all of the examples that come with AppLab which use bridge code, have the Arduino_RouterBridge added under left hand side Sketch Libraries folder in App Lab.

Thanks again!

I'm glad if I can be of assistance. This is a complex subject. This even for those of us who have a good grasp on how sketch dependencies have traditionally worked in Arduino IDE and Arduino Cloud Editor, since build profiles introduce the new paradigm of formal support for per-project management of dependencies.

Thanks for the clarification. This is a more user friendly representation of the build profile.

The dependencies are defined in the library.properties metadata file:

That is correct. When you "install" a library in an App, entries for all transitive dependencies specified by that library's metadata (and by the metadata of those dependencies, and so on...) are also added to the build profile.

This is done in anticipation of a plan to stop bundling the Arduino_RouterBridge library with the "Arduino UNO Q Board" platform in the near future.

Everytime, I start my App Lab and begin to run a program, it gives me compilation error and asks to add MsgPack.h Libray. How can I get rid of it?

  1. Open your App in Arduino App Lab.
    ⓘ Unlike Arduino IDE which installs libraries globally for availability from any sketch project, Arduino App Lab installs libraries on a per-App basis, so this process will only make the library available for use in the specific App you have open at the time you perform it.
  2. Click the button that looks like an open book with a + sign overlay, which is located to the right of the "Sketch Libraries" section heading in the left hand panel of the Arduino App Lab window.
    The "Add sketch library" dialog will open.
  3. Type MsgPack into the "Search library" field of the dialog.
  4. Find the entry in the search results for the "MsgPack" library.
  5. Select the version of the library you want your App's sketch to use from the drop-down menu in the "MsgPack" library entry.
  6. Click the "Install" button.
  7. Wait until you see a "___ installed" label appear on the entry.
  8. Click the X icon at the top right corner of the "Add sketch library" dialog.
    The dialog will close.

Now start the App. This time it should start successfully without any errors related to the MsgPack.h header file.

Why not just revert wrong decision and include msgpack always? Those questions will arise every day until you fixed UX

You mean restart the App Lab, open the app, and then tun the app.

Or

Just run the app after then installation of the library.

If your project uses an Arduino library, you install that library. Is that really asking too much from people undertaking the endeavor of creating complex embedded systems projects?

Just run the App. There is absolutely no need to restart Arduino App Lab after installing a library.

Thanks @ptillisch - got it!

I can see how managing dependencies per project is going to make things much better for Arduino Apps.

This is me thinking out loud, but honestly, I feel like the Q is best used when you have the MPU and MCU talking to each other.

That is, it’s the sweet spot for the Q - using GPIO is so easy and familiar AND I get to have a full blown processor to do what I want (runs AI models, manage a keyboard, etc).

So to me, the Arduino_RouterBridge library feels like the Bread and Butter of the Q, and assuming future boards get released that follow this MPU/MCU pairing which also use AppLab, they would also likely use this bridge code.

So to me including Arduino_RouterBridge into the Apps, sort of feels like how the Serial the library is just there when I work with Arduino code.

I see though, that simply 2 clicks is all it takes to add a dependency.
Maybe I am being lazy :slight_smile: , but I can see why some folks might express the reason to just have it bundled.

That being said, I can see the advantage of not having it bundled as you outlined here:

This is a huge advancement over the old system because it means that you don't have to worry about your project being broken when you come back to it at some point in the future due to changes in the libraries you have installed.

Anyway, thanks again and all the best!

Absolutely.

It surely will be for the majority of users. However, it is worth noting that the Arduino_RouterBridge library (and its infrastructure) is one option for communication between the MCU and MPU, not a requirement. Users are free to chose an alternative mechanism if they like.

Another advantage is in compilation efficiency. Due to the relatively limited computing resources of the UNO Q, efficiency of the sketch compilation operation is especially important. A significant component of the duration of a compilation is "library discovery", where the sketch build system searches for the library that provides each of the header files referenced in an #include directive in the sketch code, then adds the paths of those libraries to the compiler's "search path" in order to allow the sketch to be compiled as C++. When a build profile is in use (as is always the case for Arduino Apps), the paths of all libraries specified by the profile are automatically added to the compiler search path so that "library discovery" can be skipped for those libraries. However, it would not be appropriate to do the same for all the libraries that happen to be bundled with the platform.

So if the user adds an entry to the build profile for the library by "installing" it in the App, they will reduce the duration of the sketch compilation as compared to relying on a copy of the library bundled with the platform.

It is true that the user has the option of doing that, even now when the library is bundled with the platform. However, the average user is not aware that there is a benefit to doing so, and so will be likely to use the bundled library rather than making the unnecessary effort of "installing" it.


A problem with bundled libraries is that enhancements and fixes can only be shipped by doing a new release of the entire platform. The Arduino Zephyr Boards platform is a very complex project with a lot of different components being worked on by different people. So making a new release is a much more significant endeavor than making a release of a standalone library.

The current situation where the library is bundled with the platform and also distributed via the Library Manager system does allow for releases of the library to be made independently from the platform. However, those releases can only be offered to users who are using the standalone library as the dependency in their projects rather than the platform bundled library.

Apart from call(), notify(), provide(), provide_safe(), and begin() methods, are there any useful methods the libray has for programmers?

@GolamMostafa Those are the ones I am familiar with thus far.

It is worth noting that function calls are ever so slightly different depending on whether the MCU (Arduino code) is calling, or the MPU (Python code) is calling.

For example, if you want to get the return value of a function called by the MPU:

my_result = Bridge.call("My_MCU_function", 6, True) //my_result is the return value from My_MCU_function

While, if you want to get the return value when calling a function in the MCU:

int my_result;
bool success = Bridge.call("My_MPU_function", 42).result(my_result);

Here, .result() saves the return value to the provided variable, and success is set equal to true/false depending on if the Bridge.call successfully returned without error.

IMPORT NOTE: This is my understanding thus far, take with a grain of salt :slightly_smiling_face:

1. What are 6, true?

2. Do you have full prototype of Bridge.call()?

3, The value of my_result is obtained from the execution of a function that is triggered when the sender’s tag ("My_MCU_function") matches the registered tag of the receiver. For example:

MPU/Script/Python/Linux intiates:

data = Bridge.call("bme280")

MCU/Sketch/Arduino/Zephyr responds:

Tag and function registration:
Bridge.provide("bme280", read_bme280); //Registry

Function definition:

String read_bme280()
{
  float t = bme.readTemperature();
  float p = bme.readPressure() / 100.0F;
  float h = bme.readHumidity();

  String data = String(t) + "," + String(p) + "," + String(h);
  return data;
}

These are arbitrary arguments, which are passed to the "My_MCU_function" method.

Here it is:

Starting app "Project 1 - Bridge Builder - Solution - 260526"
Applied app migration for platform arduino:zephyr >0.54.1. Arduino_RouterBridge is now part of the platform and shouldn't be explicitly specified

Perhaps I am misreading the output from the console, but it looks like that Arduino_RouterBridge will stay bundled with the "platform" since the release of the arduino:zephyr >0.54.1update.

I am reading this as follows →

  1. I should not add/install the Arduino_RouterBridge in the Sketch Libraries section
  2. I shoudl not #include <Arduino_RouterBridge.h> in my .ino file

I have found that it does not seem to matter if I do both steps above, but maybe there is some reason I should not add both manually?

built-in libraries are stored in $HOME/.arduino15/internal folder:

/home/arduino/.arduino15/internal
├── ArduinoGraphics_1.1.4_b90a1c54fcf876ad
├── Arduino_HS300x_1.0.0_182fd7848a9f4724
├── Arduino_LPS22HB_1.0.2_7fe1d8dd8007ad8f
├── Arduino_LSM6DSOX_1.1.2_287085db9b88b474
├── Arduino_LTR381RGB_1.0.0_88e402623ebd3bc7
├── Arduino_Modulino_0.7.0_e9ddcb4cddf945f5
├── Arduino_RouterBridge_0.4.2_2bacd18211ad070f
├── Arduino_RouterBridge_0.4.3_456a4ab6b378b066
├── Arduino_RPClite_0.3.0_66fefd6c76c3d010
├── ArxContainer_0.7.0_007f0bb2a1cdefe3
├── ArxTypeTraits_0.3.2_d65e2aabfeed7838
├── DebugLog_0.8.4_c199e2cf6415ecc8
├── MsgPack_0.4.2_a0d4adc5044d022c
├── STM32duino_VL53L4CD_1.0.5_7e2845a3c673dda2
└── STM32duino_VL53L4ED_1.0.1_aec71b2071ab306b

built-in libraries are not shown in arduino-cli lib list --all command. But newly installed libraries in $HOME/Arduino/librariesare displayed at user Location and core libraries of Zephyr are also displayed at arduino:zephyr@0.56.0 Location:

$ arduino-cli lib list --all
Name               Installed Available Location              Description
Arduino_LED_Matrix 0.1.3     -         arduino:zephyr@0.56.0 -
ArduinoJson        7.4.3     -         user                  -
CAN                          0.3.1     arduino:zephyr@0.56.0 -
ea_malloc                    -         arduino:zephyr@0.56.0 -
RTC                          1.12.0    arduino:zephyr@0.56.0 -
SocketWrapper                -         arduino:zephyr@0.56.0 -
SPI                          -         arduino:zephyr@0.56.0 -
stubs                        -         arduino:zephyr@0.56.0 -
Wire                         -         arduino:zephyr@0.56.0 -

  • It seems to be needed to upgrade arduino-cli to show built-in libraries like Arduino_RouterBridge, Arduino_RPClit, ... and MsgPacks libraries.

These are not "built-in libraries". These are libraries that are dependencies of sketch build profiles. The term "built-in libraries" refers to the set of foundational libraries that are automatically installed by Arduino IDE on the first run of a fresh installation:

The libraries you found are instead dependencies of sketch build profiles.

I don't agree. As the folder name indicates the libraries you found are purely an internal mechanism of Arduino CLI. They will be managed solely on the basis of Arduino CLI's internal needs for them. The user should not make any direct use of the libraries found under this folder, and should not have any need to be aware of their existence. There is no reason to make this information accessible to the user.

If you want a list of the library dependencies of a sketch profile, use arduino-cli compile --dump-profile (or just look at the sketch project file content).