How to identify Arduino board with a pre-compiled library

Projects Programming
1

Hi,

I built a library, which among other things, identifies AVR boards by looking at the macro ARDUINO_AVR_xxxx (ARDUINO_AVR_UNO for Arduino Uno) and by recording the board name in a const global variable (a string) inside a header file. In this library, other .cpp files can read the value of this variable after including this header file.

It works well except when the library is pre-compiled. When it is, it will only detect boards with different processors, but not boards having the same processor (for instance Uno and Nano). Indeed, pre-compilation is done per processor and not per board, the resulting archive is to be placed in a folder named with the processor name. When pre-compiling the code for an Uno, the value of the const global variable is recorded in the archive as “uno”, and then when uploading the code on a Nano, the board is identified as “uno”.

I tried to use a non const (and therefore non static) variable, but then comes the problem of solving efficiently the multiple definition of this variable.

Do you have an idea how to solve that?

Thanks a lot for your help

2

When pre-compiling the code for an Uno, the value of the const global variable is recorded in the archive as "uno", and then when uploading the code on a Nano, the board is identified as "uno".

It's like you are saying I've const byte x =3;in my code and I would want x to be 7 if I run that compiled binary on a nano versus an uno...
This has nothing to do with "pre-compiled" libraries (or libraries at all), you compiled specific hardwired code for one machine and uploaded to a different one and expect to see differences. Why would you expect this to work?

==> Build a binary for Uno and another one for the Nano and all will be fine.

I just tried by creating a very small library:

** **MyArduino.h** **

#ifndef __MYARDUINO_H__
#define __MYARDUINO_H__

void printName();

#endif

** **MyArduino.cpp** **

#include <Arduino.h>
#include "MyArduino.h"

#if defined(ARDUINO_AVR_UNO)
const char* boardname = "AVR UNO";
#elif defined(ARDUINO_AVR_NANO)
const char* boardname = "AVR NANO";
#elif defined(ARDUINO_AVR_MEGA2560)
const char* boardname = "AVR MEGA2560";
#elif defined(ARDUINO_AVR_MICRO)
const char* boardname = "AVR MICRO";
#else
const char* boardname = "UNKNOWN BOARD";
#endif

void printName()
{
  Serial.println(boardname);
}

I put those two files in a MyArduino folder inside my custom Library folder.

then I built a very minimal sketch

#include <MyArduino.h>
void setup() {
  Serial.begin(115200);
  printName();
}
void loop() {}

when I compile and upload for a UNO Serial Monitor (@ 115200 bauds) will show

[color=purple]AVR UNO[/color]

and when I switch platform and compile/upload for a NANO I see in the monitor

[color=purple] AVR NANO[/color]

So all seems to work fine, the IDE will recompile the library if I switch the target platform and so I can have two binaries, one for Nano and one for Uno.

--- as a side note for documentation ---

you can see all the specific #define that are added at compile time based on the target board by running the command line(unix systems)

grep board= ~/Library/Arduino15/packages/*/hardware/*/*/boards.txt | cut -f2 -d= | sort -fu

on my system I get a long list

ALKS
AMPERKA_WIFI_SLOT
AVR_ADK
AVR_BT
AVR_CIRCUITPLAY
AVR_DUEMILANOVE
AVR_ESPLORA
AVR_ETHERNET
AVR_FIO
AVR_GEMMA
AVR_INDUSTRIAL101
AVR_LEONARDO
AVR_LEONARDO_ETH
AVR_LILYPAD
AVR_LILYPAD_USB
AVR_LININO_ONE
AVR_MEGA
AVR_MEGA2560
AVR_MICRO
AVR_MINI
AVR_NANO
AVR_NG
AVR_PRO
AVR_ROBOT_CONTROL
AVR_ROBOT_MOTOR
AVR_UNO
AVR_UNO_WIFI_DEV_ED
AVR_YUN
AVR_YUNMINI
BPI-BIT
CoreESP32
D-duino-32
D1_MINI32
ESP320
esp32vn_iot_uno
ESP32_DEV
ESP32_DEVKIT_LIPO
ESP32_EVB
ESP32_GATEWAY
ESP32_PICO
ESP32_POE
ESP32_POE_ISO
ESP32_THING
ESP8266_ARDUINO
ESP8266_ARDUINO_PRIMO
ESP8266_ARDUINO_STAR_OTTO
ESP8266_ARDUINO_UNOWIFI
ESP8266_ESP01
ESP8266_ESP07
ESP8266_ESP12
ESP8266_ESP13
ESP8266_ESP210
ESP8266_ESPECTRO_CORE
ESP8266_ESPRESSO_LITE_V1
ESP8266_ESPRESSO_LITE_V2
ESP8266_GENERIC
ESP8266_NODEMCU
ESP8266_OAK
ESP8266_PHOENIX_V1
ESP8266_PHOENIX_V2
ESP8266_SCHIRMILABS_EDUINO_WIFI
ESP8266_SONOFF_BASIC
ESP8266_SONOFF_S20
ESP8266_SONOFF_SV
ESP8266_SONOFF_TH
ESP8266_THING
ESP8266_THING_DEV
ESP8266_WEMOS_D1MINI
ESP8266_WEMOS_D1MINILITE
ESP8266_WEMOS_D1MINIPRO
ESP8266_WEMOS_D1R1
ESP8266_WIO_LINK
ESPea32
ESPECTRO32
ESPino32
FEATHER_ESP32
fm-devkit
FROG_ESP32
GEN4_IOD
HELTEC_WIFI_KIT_32
HELTEC_WIFI_LORA_32
HELTEC_WIFI_LORA_32_V2
HELTEC_WIRELESS_STICK
HORNBILL_ESP32_DEV
HORNBILL_ESP32_MINIMA
INTOROBOT_ESP32_DEV
LOLIN32
LOLIN_D32
LOLIN_D32_PRO
LoPy
LoPy4
M5Stack_Core_ESP32
M5STACK_FIRE
M5Stick_C
MH_ET_LIVE_ESP32DEVKIT
MH_ET_LIVE_ESP32MINIKIT
MOD_WIFI_ESP8266
NANO32
Node32s
NodeMCU_32S
ODROID_ESP32
ONEHORSE_ESP32_DEV
OROCA_EDUBOT
Pocket32
PYCOM_GPY
QUANTUM
SAMD_CIRCUITPLAYGROUND_EXPRESS
SAMD_MKR1000
SAMD_MKRFox1200
SAMD_MKRGSM1400
SAMD_MKRNB1500
SAMD_MKRVIDOR4000
SAMD_MKRWAN1300
SAMD_MKRWAN1310
SAMD_MKRWIFI1010
SAMD_MKRZERO
SAMD_NANO_33_IOT
SAMD_TIAN
SAMD_ZERO
SAM_DUE
SAM_ZERO
T-Beam
T-Watch
TTGO_LoRa32_V1
TTGO_T1
UBLOX_NINA_W10
WESP32
WIDORA_AIR
WIFIDUINO_ESP8266
WIFINFO
WIPY3

the real #define will have a leading [color=blue]ARDUINO_[/color] so that's how you get [color=blue]ARDUINO_[/color][color=purple]AVR_UNO[/color]

==> As #define, they are taken into account at pre-processing time, even before the compilation or link stage.

3

Hi J-M-L,
first of all, thanks for your reply and investigation.

My problem is the library specification is architecture specific, not board specific (Redirecting) : if I build 2 archives, one for Uno, and one for Nano, where do I put them so they are distinguished? According to the library specification, binaries have to be put inside the library folder in a folder having the architecture name. The thing is Uno and Nano have the same processor, so their binaries will be put in the same folder called "atmega328p". Then how do we specify which one must be used when the whole project is compiled? The specification does not mention a sub-folder for each board (e.g. "atmega328p\uno" and "atmega328p\nano").

4

I see.

I don’t think you can get around this without recompiling the part of the code where the #define are checked since it’s a compile time flag.

You either need to walk away from using precompiled stuff altogether and let the IDE decide when or not to recompile or devise a different strategy for example to have a sub part of your library that always get recompiled for the targeted board and only keep architecture dépendant stuff in a precompiled architecture file.

5

You are perfectly right. I will try splitting the library into two libraries, one being recompiled each time and the other being precompiled. It will not be so easy as there are board-dependant macros used a little bit everywhere.

Thanks again for your help :slight_smile:

6

Is the code so long to compile that you need to worry about precompiled library? Or is it an intellectual property thing and you don’t want to give away source code?

You could take an approach similar to HardwareSerial where you have a class with a subclass and one instance created by the library (here for Serial)

Say you have a Platform class and that class offers an API returning the platform name and spec like memory or whatever (could directly be public variables). At the end of this class you instanciate a global variable.

Then in the code where you use it you replace your variable (or define) platformName by platform.name (where platform would be the one instance of the Platform class)

Your library could only need to know about this extern instance that will be resolved at link time I suppose.

7

Is the code so long to compile that you need to worry about precompiled library? Or is it an intellectual property thing and you don't want to give away source code?

It is an intellectual property thing.

You could take an approach similar to HardwareSerial where you have a class with a subclass and one instance created by the library (here for Serial)

Say you have a Platform class and that class offers an API returning the platform name and spec like memory or whatever (could directly be public variables). At the end of this class you instanciate a global variable.

Then in the code where you use it you replace your variable (or define) platformName by platform.name (where platform would be the one instance of the Platform class)

Your library could only need to know about this extern instance that will be resolved at link time I suppose.

Thanks for this solution. I will have a look to that and try to implement it.

8

jacovalseur:
It is an intellectual property thing.

I assume by this comment that your are wanting to create a closed source library that is likely to be used in a closed source product.

Creating a product using closed source is not as easy as one would think when linking it with other open source due to licensing restrictions.

Keep in mind that depending on which libraries you use, you may not be able to use them in a product with closed source or may be forced to open up your source code.
i.e if the project/product uses a GPL v3 library, then you cannot use the library unless you open up all your source code since gpl v3 code can only be used with other open source code.
If a library uses CC BY SA 3.0 or lower there is no way to legally use it unless all the code is also licensed as CC BY SA which can't be done with Arduino since there are Arduino core libraries and gcc or avr libC code that is LGPL.
If a library uses CC BY SA 4.0 there is no legal way to use it with / link with closed source as the only way to make it compatible with other non CC BY SA licensed open source licenses is to use the 1 time convert clause to convert it to gpl v3 and then all the gpl v3 rules apply.

Also, even if you use only LGPL 2.1 code which does allow using with closed source, according the LGPL license, you must provide a way for any user to modify and update any of the LGPL code used in the image.
So users will need to be able to rebuild and update the firmware image in your device.
This is a very difficult and sometimes impossible thing to do.
While I know that Arduino has made some changes in newer versions of the IDE to supposedly be able tolink against pre-compiled libraries, I've not ever done it so I'm not sure if it is really possible to create a package that allows a user to update any/all the other non closed source LGPL / GPL code to update their firmware.

Here is potential non obvious side effect that can happen when closed source is used to make a pre-compiled library that is offered for sale / distributed.

Say person creates a closed source library from all of their own code that uses no other code.
(unlikely in Arduino, but suppose it could be done)
They then offer it for sale or even decide to distribute it for free
(whether there is a charge for the library does not matter)
No issues so far and people can purchase/obtain the library.

However; the issue becomes can anybody legally use the library in their product?

The issue is that while creating the pre-compiled library and distributing it didn't violate any licensing, linking that library into a larger project may violate the licensing agreements of other code used in the image.
If that is the case, the library can not legally be used in the same image with code that has an incompatible license.

i.e. all licenses of all the individual components linked together to create the final image must have compatible licenses and any one component may put restrictions on all the other pieces of code that are more restrictive than the original license of that code.
Such a restriction may even preclude use of that code in the project / product.

i.e. it is possible to legally create and distribute a library that nobody can legally use.

Just some examples of how tricky using open source is when trying to use it with closed source.

--- bill

9

Hi Bill, thanks for your detailed thoughts about closed source library.
I already went through all this and yes it is very complicated. The only way I found to solve this is to create a custom license (EULA) that the user must accept and which restricts its use of the library.

If that is the case, the library can not legally be used in the same image with code that has an incompatible license.

If he accept the terms of the license (= if he uses the library), the user is then not allowed to use my library with any incompatible copyleft open source licenses that compromise or interfere with my company's property rights or require my company to disclose any source code. I clearly inform users about this restriction in the library documentation as well. A license is a contract that the user accept. So it is up to the user itself to know what he is doing. If he uses multiple libraries, it means he accepted several contracts. If he still chooses to mix incompatible licenses in a project and make this project official, then I suppose he may be forced at some point by some authority to remove libraries from his project so that there is no more incompatibility.

While I know that Arduino has made some changes in newer versions of the IDE to supposedly be able tolink against pre-compiled libraries, I've not ever done it so I'm not sure if it is really possible to create a package that allows a user to update any/all the other non closed source LGPL / GPL code to update their firmware.

I am using pre-compiled libraries and this feature works well. It is possible for the user to independantly update any other library (LGPL or other) without having to update the pre-compiled library.

10

Arduino was created in the spirit of open source.

In general, the available program memory in Arduino is small enough that there is little most people can do, in terms of "intellectual property", that could not be rather easily and quickly duplicated by someone with modest skill and motivation.

Are you really so confident of the sheer brilliance of your efforts, that you must jump through all these hoops to keep the end result a secret?

11

jacovalseur:
If he uses multiple libraries, it means he accepted several contracts.

There is no "if" about using multiple libraries when building code using the Arduino environment.
At a minimum there will be some gcc startup code, and some Arduino core library code and likely some additional Arduino libraries and possibly some 3rd party library code.
so for sure LGPL 2.1 licensing will apply.

What concerns me is the use of language and the potential mindset behind a statement like this:

If he still chooses to mix incompatible licenses in a project and make this project official, then I suppose he may be forced at some point by some authority to remove libraries from his project so that there is no more incompatibility.

This seems to be taking a view that violating a s/w license is ok until you are caught and forced to comply.
That isn't how s/w licensing works.

Like I said it is not difficult to create a closed source library but it is possible to unintentionally create a closed source library that very few or even no one can legally use.

The creators / author of the library typically don't have any liability for is misuse with use other non compatible licensed s/w.
The user of the library it is usually fully responsible.
However, there can be some potential legal blow back on the creator of the library if the library can never function without being used in conjunction with another s/w component that is not license compatible.
i.e. a close source library that can only function when linked against a gpl v3 library.

jremington:
Arduino was created in the spirit of open source.

On that note, there is an increasing number of 3rd party libraries that are taking an extra step of using licenses such as gpl v3 to preclude using any closed source in the same image as the library.

jremington:
In general, the available program memory in Arduino is small enough that there is little most people can do, in terms of "intellectual property", that could not be rather easily and quickly duplicated by someone with modest skill and motivation.

Are you really so confident of the sheer brilliance of your efforts, that you must jump through all these hoops to keep the end result a secret?

This becomes about a trade secret.
The IP in a trade secret is not protected like a patent.
As jremington pointed out, any person can legally reverse engineer the code and re-implement it and you have no recourse.

If the code implements a patented invention it does not matter if the implementation is released as open source or even reverse engineered.
It is always protected by the patent.
When the invention is protect by a patent, it is protected no matter how the code is written and there is no need for secrecy.
i.e. anybody that implements the functionality is subject to patent licensing and restrictions no matter how it is implemented.

--- bill

12

.

There is no "if" about using multiple libraries when building code using the Arduino environment.
At a minimum there will be some gcc startup code, and some Arduino core library code and likely some additional Arduino libraries and possibly some 3rd party library code.
so for sure LGPL 2.1 licensing will apply.

LGPL 2.1 license is not viral, even after static linking. Licenses of 3rd party libraries will apply as well. If a user want to build an Arduino project with 3rd party libraries, it is his responsibility to make sure the terms of all the libraries licenses are respected and compatible.

I made my library license compatible with LGPL 2.1 but not with GPL which is viral. So user will not be allowed to use my library in combination with the SD library for instance.

This seems to be taking a view that violating a s/w license is ok until you are caught and forced to comply.

I never said it was ok to violate a license, thanks not to misinterpret. I just mentioned the case where it would happen, because it is a scenario to anticipate when writing its own license and trying to protect its own work.

Arduino was created in the spirit of open source.

True. Also true that Arduino started a move to enable closed source code to be used in Arduino environment. Indeed, LGPL 2.1 requires other libraries statically linked to give the user a way to modify/update the LGPL code and to rebuild the whole work. This is what pre-compilation offers.

Are you really so confident of the sheer brilliance of your efforts, that you must jump through all these hoops to keep the end result a secret?

I am confident on the time spent working on the library, on the value I add, and on the fact I need to earn a living. Jumping through hoops is what entrepreneurs do every day

13

jacovalseur:
I never said it was ok to violate a license, thanks not to misinterpret. I just mentioned the case where it would happen, because it is a scenario to anticipate when writing its own license and trying to protect its own work.

I'm sorry if I offended you.
From the words written, your actual view wasn't 100% clear, which is why I said "This seems to be taking a view..." with "seems" being the important word, in that I saw what the written words said and meant but wasn't sure that is what you really meant.
The literal interpretation of your written words implied that it was ok until it wasn't when somebody came around and enforced the rules.
The key phrase being ""he may be forced at some point by some authority", which implies that there is some period of time prior to that point in time where "some authority" enforces the licenses where the project/product that builds a code image using incompatible licenses would exist and potentially be distributed.
i.e. the use of the word "may" specifies that he may or may not ever be forced to do so and then only when some authority comes along and forces him to correct things.

When the licenses are not compatible the user of your library is never allowed to do that type of mixing for a project / product he makes available to others.

I'm super curious, what is the general nature of this library?
Can you give us a couple of sentences of a high level description?

--- bill

14

I would also be interested in the answers to bill's questions, and wonder what you think your customer base might be.

People posting on this forum generally seem unwilling to pay more than the value of a cup of coffee for a sensor, let alone pay for software.

15

I am targetting people willing to pay for a product which save them dev time and cost, in other words mainly pros and businesses.

I aim at selling a computer software which communicates with arduino boards. This software would solve very specific user needs that I won't detail here. The purpose of the library is to efficiently handle communication and data transfer with the software. However, in order to prevent competitors from recreating a similar computer software and just distributing my library with their product, I had to close the library source code to hide the communication protocol.

16

The typical objection you’ll get from Pros and businesses is what happens if you get busy doing something else, if you are no longer interested in maintaining that piece of code because you have a much more interesting project to work on or if you break your wrist...

A critical dependency to something you can’t control and limits your other library options is just building technical debt. Most company would stop there and assess that risk. This is usually addressed by including release of source code in the contract in case you fail to deliver a patch in a certain time but it sets a high bar to close the sales.

It seems very niche market when there are so many free and open source options to choose from. Hope you did your homework on the value prop and how you plan to address your target market.

17

Going back to the very original question of
"How to identify Arduino board with a pre-compiled library"

The short answer is I don't believe it is possible.
You can identify an architecture at compile time and potentially a specific CPU runtime but not an Arduino board.
This is because Arduino board types are an "Arduino" thing handled by the build tool so there is nothing you can look at run time including fuses to determine a board type since the same identical processor can be used on different boards
Each board can wire things up any way it wants as long it has a corresponding boards.txt entry and associated variant arduino_pins.h file.

The only way to identify a board type is to look at compile time "things" that are in header files or created by the Arduino build tools.
Those can be macros created by the compile recipes in the platform's platform.txt which can also be fed some variables that come from the board entry in the boards.txt file, they can come from the boards variant pins_arduino.h file or they could come from a macro in Arduino.h that references macros in the board's variant pins_arduino.h file.

I've done board specific stuff for a graphics library I have. (openGLCD)
It is quite painful due to the way the boards.txt and platform.txt defines work in combination of the arduino build tools.
Something that sounds simple like coming up with a printable board name is very difficult and sometimes impossible, particularly on the arduino.cc platforms since the arduino.cc platforms create a board specific define symbol name but never creates a macro/symbol with one name that gets the name of the board.
i.e. there is no way to get a name from the build system from one macro/symbol name so you have to create it based on looking at many compile time things.
It is easier on other platforms like the chipKIT platform as they hand the board name as a define symbol that is a constant name to the code being compiled, but that isn't the case for arduino.cc supplied platforms like avr, and due that uses a different symbol name for each board type.

Hard-coding for specific macro names for each board types is really ugly and a total pain but is not too bad if there is only a single variant for a given processor, but that is often not the case, plus a user can create their own board types if they want to.
Once there are multiple boards for the same processor, things start to get really complicated and sometimes it can be impossible to identify even at compile time.

In some cases you must reach deep down into macros in side the arduino_pins.h file like looking at the values of symbols like the analog pins, the specific value of certain function like macros. For example: the macro analogInputToDigitalPin(0) can be used to distinguish between certain variants of the mighty1284 platform.

But... and there is the big but.

You can't do any of these types of compile time board detection things if the code shipped as a pre-compiled object and the code is expected to run on multiple Arduino boards.

There is an even bigger issue if the source is closed.
This is because all the Arduino supplied core code, and variant code, along with some macros in Arduino.h are all licensed LGPL 2.1+
This means that if you use any of those macros and compile them into your code to use in your library to do checks runtime vs compile time but ship it precompiled, the user must have a way to update those macros.

There are a ton of macros/symbols and function like macros from Arduino.h that fall into this category.
Some not so obvious like some math functions including min(), max(), round() and several others.
Lot of arduino specific macros to handle bit manipulation, and some macros to handle pin to port and bit mapping.
Use any of those macros in your closed source code and ship a pre-compiled object, then you must provide a way for the user to update them inside that precompiled object, which is not possible without the source code.

The only way to ship a pre-compiled closed source library would be make it stand alone code and not use any macros or inline functions from the gcc tool set, platform libraries like avrLibC, arduino core, variant files, or even other libraries.

--- bill

18

Thanks J-M-L and Bill for your answers. First of all, I wish you guys a very happy new year 2021 :slight_smile:

It seems very niche market when there are so many free and open source options to choose from. Hope you did your homework on the value prop and how you plan to address your target market.

Only the future will tell if I did my homework correctly, but I know there is interest in what I propose.

The typical objection you'll get from Pros and businesses is what happens if you get busy doing something else, if you are no longer interested in maintaining that piece of code because you have a much more interesting project to work on or if you break your wrist...

Well if the project really works, my idea is to make it grow, to hire developers and employees. So if I can get to that, my wrist should no longer be a problem :slight_smile:

You can't do any of these types of compile time board detection things if the code shipped as a pre-compiled object and the code is expected to run on multiple Arduino boards.
...
This means that if you use any of those macros and compile them into your code to use in your library to do checks runtime vs compile time but ship it precompiled, the user must have a way to update those macros.

You are right. So these past 2 weeks, I kept on studying at the GPL and LGPL licenses in Arduino environment and how those could affect my license choice.

If I keep on trying to use a closed-source pre-compiled code while still be able to identify boards, a possible solution I came in is to spit my library into 2:

  • a 1st closed-source pre-compiled library which would use agnostic functions, with no macro from other libraries. It would include headers only from the 2nd library
  • a 2nd open-source library under a permissive LGPL-compatible license, which would act as an interface for the 1st library. It would embed the use of all macros and functions from other LGPL libraries in .cpp files.

The problem with this solution is the AVR and ARM toolchains. Those toolchains are a mix of libraries using a mix of GPL and LGPL licenses. GCC is GPL and has the GCC Runtime Library Exception. However this exception does not seem to include the insertion by the user of GCC macro such as AVR..._ in its code... And I need to use such macros. Also I use linker variables, and the linker is binutils (not clear if it is GPL or LGPL).

I will keep on making a bit of search, but I might end up being forced by the toolchains to make my whole lib open-source.

19

Using wrappers to get around GPL certain licensing requirements for certain closed source products is one of the things that drove Stallman to create GPL v3.
IMO, GPL v3 ended up going too far for some people, particularly with some of the requirements related to patents.

I don't think using simple preset macros like AVR is an issue.
The concern would be for function like macros and gcc support or library functions that get inlined by the compiler.
Things like certain math functions like max(), min(), floor() , etc... can be inlined by the compiler.
So you have to be careful as gcc can substitute its library functions inline unless you explicitly tell it not to.

In the case of Arduino there are macro like functions for some of the port mapping macros:
digitalPinToPort(), digitalPinToBitMask() portOutputRegister() portInputRegister() etc...
Or math functions like max(), min(), abs(), round, constrain(), radians(), degrees(), sq()
since Arduino redefines them as macros rather than let arduino code use the versions that come with the compiler.

Still, seems like a whole lot of work for a pretty small sized project. (at least codewise - given the small flash size on AVRs)
If the code is that unique, I would be concerned if a Patent were not involved.
The reason being, anytime there is an actual reasonable sized market for something usually there ends up being competition.
The patent would protect the IP implemented by the code against others figuring out what the code is doing and writing their own implementation to compete against you.

--- bill

20

Wishing you well for 2021 too!!