conflicting library versions

Hi all,

I am currently using arduino-cli, and am writing to ask if it is possible to disable the 2 default library include paths
(in Windows):

  1. C:\Program Files (x86)\Arduino\libraries
  2. %userprofile%\Documents\Arduino\libraries

The reason is that for some of my projects i am using older versions of some libraries because with the latest ones my code doesn't compile anymore (library interface changed). I don't want to update my code to support the newest libraries because this does not protect me from the problem possibly happening again when a library gets updated further.

Instead, i would like to keep the relevant libraries inside the project source tree and use the --library option to include them; but if i do so, i get compile errors because the 2 directories above are always included by default, thus ending in conflicting/duplicate declarations.

A temporary (and horrible!) solution i found was to leave the default library directories empty by relocating the system and contributed libraries somewhere else, but this adds extra work to keep the libraries updated (for my newer projects) since the library manager doesn't work anymore.

An additional command line option to disable these 2 default locations or a CLI version requiring to mandatorily specify also the 2 aforementioned library paths would be perfect.

Does anybody know if there is a secret/hidden command option or configuration to alter to accomplish that?
Thanks and regards

nomadic68:
I am currently using arduino-cli

...

  1. C:\Program Files (x86)\Arduino\libraries

Instead, i would like to keep the relevant libraries inside the project source tree and use the --library option to

This is not clear to me. Are you using Arduino CLI?:
https://arduino.github.io/arduino-cli/latest/
or the command line interface of the standard Arduino IDE?:

The reason I ask is because Arduino CLI only uses the libraries subfolder of the Arduino IDE installation folder in the event it is located in the Arduino IDE installation folder, which is an unusual circumstance at this time when Arduino CLI has not yet been integrated with Arduino IDE.

The other thing I don't understand is the mention of the --library option. Neither program has such an option. Arduino CLI does have a --libraries option.

Hi,

I am using the Arduino CLI, not the command line interface of the standard IDE:
my aim is to have a better control of the whole source tree (including libraries) and the compiler options.
(that is, like i had with Atmel Studio when it was working; at a certain point the support for the ARM boards broke and never got fixed, so i moved away from AS7 and landed to Visual Studio Code and Arduino CLI).

And yes, I was meaning the --libraries options of Arduino CLI, sorry for the typo.

Would it be possible to make a change request to the CLI developers to allow better control of the libraries folders? How?

Thanks

OK

nomadic68:

  1. C:\Program Files (x86)\Arduino\libraries

As I mentioned before, this folder will only be used by Arduino CLI if it's bundled with the Arduino IDE at C:\Program Files (x86)\Arduino. This is in preparation for the planned complete replacement of arduino-builder with Arduino CLI as the engine behind the Arduino IDE's GUI. So you can simply use a standalone installation of Arduino CLI to avoid the use of this folder.

nomadic68:
2) %userprofile%\Documents\Arduino\libraries

Arduino CLI always uses the libraries subfolder of the user directory. %userprofile%\Documents\Arduino is the default user directory, but you are welcome to set it to any location you like. You can learn more about that here:
https://arduino.github.io/arduino-cli/latest/configuration/
Note also that configuring Arduino CLI has become even more convenient with the addition of the ability to control the configuration settings via arduino-cli config:
https://arduino.github.io/arduino-cli/latest/commands/arduino-cli_config/

You can have a separate configuration file for each project if you need different settings.

nomadic68:
and the compiler options.

You can inject arbitrary compiler flags using the arduino-cli compile --build-property flag:
https://arduino.github.io/arduino-cli/latest/commands/arduino-cli_compile/#options
Some detailed discussion about that here:

nomadic68:
Would it be possible to make a change request to the CLI developers to allow better control of the libraries folders? How?

Sure, though it seems to me that the tool already provides all the functionality you seek. You can learn about submitting feature requests and bug reports here:
https://arduino.github.io/arduino-cli/latest/CONTRIBUTING/#issue-reports

@Pert: 1000 thanks for the useful indications, now i have a lot to work on!
You are right, i have downloaded the Arduino-CLI and started using it without uninstalling my Arduino IDE, so the CLI found the boards and the libraries automatically. I will look at the configuration options you pointed me to in order to understand them better.
I also came across the PRO version of the IDE, i got curious, downloaded it and gave it a look: surprisingly, the PRO IDE didn't find the normal IDE folders. If i get this one to work the way i want, it will save me from the hassle of studying how to set up VS Code instead. Wow!

nomadic68:
@Pert: 1000 thanks for the useful indications, now i have a lot to work on!

You're welcome. Please let me know if you end up having any other questions or problems.

nomadic68:
i have downloaded the Arduino-CLI and started using it without uninstalling my Arduino IDE, so the CLI found the boards and the libraries automatically.

There's no problem with having Arduino CLI and Arduino IDE installed at the same time. I always have both installed (as well as Arduino Pro IDE). By default, Arduino CLI does use the same user, data, and download directories as the Arduino IDE, but you can always configure it to use other directories if that's a problem for you. The only time the Arduino IDE being installed has special implications for Arduino CLI is if Arduino CLI is installed to the same folder as the Arduino IDE, but that's easy enough to work around simply by moving Arduino CLI somewhere else.

nomadic68:
If i get this one to work the way i want, it will save me from the hassle of studying how to set up VS Code instead. Wow!

I'm glad if the Arduino Pro IDE provides for your needs. Arduino Pro IDE actually uses Arduino CLI under the hood so you will find the Arduino CLI configuration file information also applies to the Arduino Pro IDE configuration.

Hi Pert/all,
Sorry for the long silence, sometime work becomes crazy...

I tried both the arduino-CLI and the new IDE 2.0 and found something different than what Pert wrote in his replies to my question; to be absolutely sure, i repeated the tests in a clean virtual machine (win10 x64) and got the same results; my installation is identical on the real PC and the virtual machine:

Arduino IDE 1.8.13 installed using Windows installer (not the marketplace app!), all default directories.
Added SAM and SAMD boards packages.
Added Adafruit MCP23017 library just to have one (installed via library mgr it ended in the sketchbook library folder).
Arduino-CLI 0.18.0 in the same directory of 1.8.13 IDE
Arduino IDE 2.0 beta 4 in C:\Arduino2IDE_b4

The outcome of the tests:

Arduino-CLI:
I didnt install any boards nor libraries from Arduino-CLI.
the CLI lists the same boards as the IDE 1.8.13 (correct)
the CLI lists only the custom library (MCP23017) i installed from the IDE; the "system" libraries (SD, SPI, Ethernet...) are not there. Running the "lib update-index" command doesnt change the result (different from Pert wrote)

Arduino 2 IDE:
it sees the same boards as IDE 1.8.13 (correct)
when i click on "include library" in the new empty sketch, it sees only: SPI HID SoftwareSerial EEPROM Wire as "system" libraries, and the MCP23017 user library.
when i install Ethernet library it gets installed in the sketckbook library folder.

These are really from 1.8.13 IDE because if i clean the virtual machine and run CLI or IDE 2.0 without prior installation of the 1.8.13 IDE i see no boards and no libraries at all.

I also tried to download the Ethernet library from the CLI, and it has been installed in the sketchbook libraries folder (%USERPROFILE%\Documents\Arduino\libraries).

Is there something broken?

PS: i gave a try also to the new Microchip Studio: it still has the same problems as the old Atmel Studio about compiling for Due.

Best regards

nomadic68:
the "system" libraries (SD, SPI, Ethernet...) are not there. Running the "lib update-index" command doesnt change the result (different from Pert wrote)

The libraries you have mentioned are actually of two different types. You might be used to always having SD and Ethernet pre-installed because they come with the installation of the classic Arduino IDE, but neither Arduino CLI nor Arduino IDE 2.x come with any pre-installed libraries, so you must install these.

As for SPI, this is a platform bundled library. By default, arduino-cli lib list does not show these libraries. But it will if you add the --all flag to the command. You can also add the --fqbn flag to show only the platform bundled libraries for that specific board. Documentation here:
https://arduino.github.io/arduino-cli/latest/commands/arduino-cli_lib_list/

With that cleared up, is there anything else that is not as you expect?

Hi Pert, thank you for your quick reply which clarified my misunderstanding of the Arduino IDE organization.

If i remember well, with a previous version of the arduino-cli in the same directory as the IDE executable i was getting the IDE pre-installed libraries listed even without the --all flag.
Never mind: the current behavior is perfect for me, because at the beginning of the thread i was asking if there was a way to exclude the 2 default directories, that is:

  1. c:\Program Files (x86)\Arduino\libraries
  2. %USERPROFILE%\Documents\Arduino\libraries
    in order to have a more AS7-like management of my projects, with a local copy of all required libraries in the project folder (still have to find how to get rid of 2) folder, though).

Sorry if i have irritated you, it is clear that i missed to read the relevant parts of the online documentation, relying only on the help text.

BTW, is there a way to get an offline copy like PDF or similar, so i can read it thoroughly while i travel?

Thanks and regards!

nomadic68:
If i remember well, with a previous version of the arduino-cli in the same directory as the IDE executable i was getting the IDE pre-installed libraries listed even without the --all flag.

That was indeed the way Arduino CLI worked two years ago, but it was changed to the current behavior in the 0.3.7 release.

nomadic68:
the current behavior is perfect for me

I'm glad to hear it!

nomadic68:
Sorry if i have irritated you, it is clear that i missed to read the relevant parts of the online documentation, relying only on the help text.

Not at all. I love talking about this stuff, probably a lot more than most people have any desire to hear about it.

nomadic68:
BTW, is there a way to get an offline copy like PDF or similar, so i can read it thoroughly while i travel?

The source of the documentation you see hosted at Arduino CLI is here:
arduino-cli/docs at master · arduino/arduino-cli · GitHub
The "Command reference" section of the documentation is not there, and neither the "gRPC reference" section because those are generated from the Golang code of Arduino CLI. But the Command reference is identical to the command line help you get from arduino-cli and there's a good chance you have no interest in the gRPC interface.

Even though it's the raw markup for the documentation in those files in the docs folder, Markup is quite readable even in its raw state, and there are also plenty of applications that will render it for you, VS Code's Markdown preview for example.

If you want to get really fancy, you can serve a local copy of that website from your own computer. The instructions for that are here:
https://arduino.github.io/arduino-cli/latest/CONTRIBUTING/#working-on-docs
You do need to install the tools, but after that, it's simply a matter of running the command task docs:serve from the folder containing the Arduino CLI repository. I use that system all the time while doing development on the documentation.

Even though this documentation is hosted in the Arduino CLI repository, much of the information applies to all the official Arduino development software (Arduino IDE, Arduino IDE 2.x, Arduino Web Editor). The reason is that Arduino CLI is used under the hood by all those applications to handle non-GUI tasks. It's definitely not something the average beginner should be exposed to, but for those of us who like to dive a bit deeper, there is some very interesting and valuable information there.

nomadic68:
Thanks and regards!

You're welcome. I'm glad if I can be of assistance.

1 Like

btw I have been created a package manager for any Arduino project which takes care the versions also (so if any conflict is in place with dependencies it warns you etc…).
It’s super simple to use but still makes your project mainntainable and portable!

You can find it here: GitHub - ksrichard/apm: Arduino Package Manager

1 Like

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.