I am proposing to extend this a bit to make it possible to use existing (e.g. cmake) c and c++ libraries mainly unchanged just by adding a library.properties file with some additional information to make them usable in Arduino:
possiblility to specify one or many separate header file directories, dependent & independent on the platform: most often existing libraries have all their header files in a separate includes directory. It is my gut feeling that just with this we have covered 80% of the cases.
possibility to exclude directories or files dependent on the platform: currently all files in the src directory are recursiveliy compiled. This is giving error messages if there are any files that must be ignored for the current platform. With the current approach, we need to change all these source files and this approach is breaking down if the project includes assembler files.
I have converted quite a few (audio) libraries and it is managable if you have a simple library but for more complex cases things are starting to get messy quite quickly.
I also noticed that the ESP32 Arduino started to blow up it's core by adding things like a mp3 codec, tensorflow etc, I assume just because this is easier than to change the existing code into an Arduino library.
What components would we need to extend to make this work ?
Are there any other ideas to improve the situation ?
Hoping for a fuitful discussion and I would be willing to contribute...
I don't know the current system in detail, but would be nice that the build process would be based in cmake, keeeping the same code generation and magics than now, and transparent for the users.
But with better build options for the libraries, and even for the users. I don't know if this would be too difficult.
That would be great! Not being able to use existing C++ libraries is one of the main reasons that drive me away from the Arduino platform for any serious project these days. CMake has become the de facto standard for most C++ projects, and libraries built using CMake are simply incompatible with the way the Arduino builder expects the source code to be structured.
For me the question then becomes: Is it worth reinventing an entire build system and further complicating the existing Arduino library format, or should CMake-based libraries be supported directly by invoking CMake as part of the Arduino build process? In my opinion, the latter option makes more sense, and carries a significantly lower maintenance cost for both library developers and the Arduino team.
For context, last year's Jetbrains Developer Survey shows that 74% of embedded C++ developers used CMake as their build system: https://www.jetbrains.com/lp/devecosystem-2023/embedded/#pm. This is of course a biased sample, but it demonstrates the relevance of CMake-based projects and libraries. Furthermore, most popular embedded platforms and libraries like the ESP-IDF, Raspberry Pi Pico SDK, STM32CubeIDE, ETL, FreeRTOS, Zephyr, etc. are either built using CMake themselves, or allow users to select CMake as their build system of choice.