Telling the IDE which header file to use..

Over a period you ( atleast me ) end up with many versions of the same header file. Notorious among them are the ones like LCD / SPI / Wire / SD etc.

When you compile using one of these the IDE reports about ALL available versions and the one that it uses right now. Possibly that's not what I want.

One option is to make sure that there is only one location for the library file ( inside MyDocuments on the Windows install drive ).Pointed via Preferences dialog box. Are there any more locations where these files can reside ?

Actually came across this issue when I added the Teensy 3.5 board to the IDE and it brought alongwith it a host of libraries. And some code written for the Teensy failed to compile as the library being used was from the Arduino locations.

Short of having only the required libraries just wanted to know if there is any way to discipline the IDE to compile only with the library I want it to compile ??

Just delete or rename the ones you don't want.

You can also give the #include the full path name.

It is unusual that Teensy interferes like that. In most cases the Teensy version of the library is superior.

I know deleting unwanted libraries is the obvious way forward. But the problem arises when there are multiple instances - not sure which to keep . Of course the IDE uses the libraries in the Sketchbook Location which I have entered in preferences. But its quite possible that I need to use a different variant ( like the Teensy one in the case below ). So how to handle that situation ? Just look at the screen shot of a failed compile below :

Multiple libraries were found for "TimeLib.h"
Used: C:\Users\RRNathan\Documents\Arduino\libraries\Time
Not used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Time
Not used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Time
Not used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Time
Not used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Time
Multiple libraries were found for "SPI.h"
Used: C:\Users\RRNathan\Documents\Arduino\libraries\SPI
Not used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\SPI
Multiple libraries were found for "SD.h"
Used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\SD
Not used: C:\Program Files (x86)\Arduino\libraries\SD
Multiple libraries were found for "Wire.h"
Used: C:\Users\RRNathan\Documents\Arduino\libraries\Wire
Not used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Wire
Using library SD at version 1.0.8 in folder: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\SD
Using library Wire at version 1.0 in folder: C:\Users\RRNathan\Documents\Arduino\libraries\Wire
Using library Time at version 1.5 in folder: C:\Users\RRNathan\Documents\Arduino\libraries\Time
Using library SPI at version 1.0 in folder: C:\Users\RRNathan\Documents\Arduino\libraries\SPI
Error compiling for board Teensy 3.5.

In most cases the Teensy version of the library is superior and works just fine on all other Arduino variants. The Teensy 2.x has an AVR chip just like the other 8-bit Arduinos you are used to.

I have never installed other boards but why do you have a Wire library and an SPI library in the Documents\Arduino\libraries directory.

sterretje:
I have never installed other boards but why do you have a Wire library and an SPI library in the Documents\Arduino\libraries directory.

Yes why do I have ??!! These are things that must have happened when I installed the Teensyduino.

OK so it looks like the best way is to have all required libraries in central place that I choose in the Preferences dialog and delete the rest.

But better still if I have the flexibility of specifying the full path in the pre processor directive #include , it will be better. I tried doing it as below in different formats …

#include C:\Users\RRNathan\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\libraries\ESP8266WiFi.h

#include "C:\Users\RRNathan\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\libraries\ESP8266WiFi.h "

#include <C:\Users\RRNathan\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\libraries\ESP8266WiFi.h>

But the IDE simply refuses to go by my directive and stops compiling !? How does one go about giving the full path to a library ??

The include priority system is pretty complex. I tried to document it here but I think there might be a couple errors:

Generally the files in the sketchbook have priority. The reason for this is so that you can put customized versions of the libraries there that will override the stock versions. Then Library Manager came along, whic installs to the sketchbook. The problem with this is hardware packages commonly have to bundle modified versions of the standard Arduino libraries but once you updated one of those standard libraries via Library Manager the hardware package override no longer worked. The workaround the Arduino developers came up with is to use architecture (as defined in the library.properties file) matching as a factor in determining include priority. Unfortunately that still doesn't help when the library in the sketchbook has the same architecture value as the hardware package, as is the issue you encountered (last time I checked TeensyDuino used the avr architecture folder for even the non-AVR Teensys).

Mogaraghu:
#include C:\Users\RRNathan\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\libraries\ESP8266WiFi.h

That's definitely wrong. As for the others, I don't know why it doesn't work. I remember seeing posts from at least one forum member here who said they use absolute include paths. I tried it with the EEPROM library and it works fine, I believe because that's a header only library. I tried it with the Stepper library and I got undefined reference errors when compiling. Same results with the double quotes or angle brackets include syntax. Maybe MorganS can tell us how to make it work.

Mogaraghu:
Are there any more locations where these files can reside ?

  • C:\Program Files (x86)\Arduino\libraries
  • C:\Program Files (x86)\Arduino\hardware\arduino\avr\libraries
  • C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries - actually 3rd party hardware packages should never be installed to the Arduino IDE installation folder but TeensyDuino does it anyway for some reason.
  • C:\Users\RRNathan\AppData\Local\Arduino15\packages{vendor}\hardware{architecture}{version}\libraries
  • C:\Users\RRNathan\Documents\Arduino\hardware{vendor}{architecture}\libraries

pert:
The include priority system is pretty complex.

Why? If arduino ecosystem is intended for beginners, it should be SIMPLE. There shouldn’t be any need for a priority system. There should be only one system-wide (or at least user-side) version of a lib.

Any custom libs bundled with packages should be bundled within that package’s directory, and be invisible to all other sketches and packages.

Then Library Manager came along, whic installs to the sketchbook. The problem with this is hardware packages commonly have to bundle modified versions of the standard Arduino libraries but once you updated one of those standard libraries via Library Manager the hardware package override no longer worked.

Unbelievable. Library Manager should not be able to overwrite packages that contain modified libs, packages with modified libs should NOT be able to overwrite the stock libs. That’s insane. They should be required to put their bundled versions in their OWN folder, and not touch the stock versions.

Library Manager shouldn’t touch anything in Documents folder. No other Windows software puts libs in Documents! Arduino is installed to Program Files, as it should be. Any libs added by Library Manager or a 3rd-party package should go into Program Files, or another system folder. Not Documents!

3rd party hardware packages should never be installed to the Arduino IDE installation folder

If they are mods of standard libs, i agree-- they should be stored elsewhere-- but not in Documents! AppData seems a good location, except then it’s exclusive to that user. Dumb! If the admin installs something for ALL users, it wouldn’t go into one user’s appdata folder. All libs (which aren’t mods of standard libs) should go into a system-wide folder, accessible to all users.

No libs should go into appdata, unless installed by a non-admin, accessible only to that user.

Library Manager should scan all libs, system-wide and user-specific, and report the presence and location of all of them, and allow users to UNINSTALL any one of them.

Back in the day, Microsoft created dll hell, making life a nightmare for system admins and developers. Thank you Arduino for giving us arduino hell.

bayviewboom:
If arduino ecosystem is intended for beginners, it should be SIMPLE.

It is simple, to the user. That doesn't mean it should or can be simple under the hood. All a beginner needs to do is install the library (if not already installed) and the add the #include directive to their sketch. They don't need to know anything about the include priority system.

bayviewboom:
There shouldn't be any need for a priority system. There should be only one system-wide (or at least user-side) version of a lib.

That would be very limiting and the users would be worse off for that restriction. How would that be better. The priority system allows us to install updates of the libraries bundled with the Arduino IDE as new versions are released. It allows us to write our own custom replacements for the bundled libraries. It allows hardware packages to provide their own custom replacements for the bundled libraries. That's extremely useful

bayviewboom:
Any custom libs bundled with packages should be bundled within that package's directory

They are.

bayviewboom:
and be invisible to all other sketches

What does that mean? What are "all other sketches"?

bayviewboom:
and packages.

They are.

bayviewboom:
Unbelievable. Library Manager should not be able to overwrite packages that contain modified libs

I said override, not overwrite. Library Manager does not overwrite the bundled libraries. And even if you meant to write "override", I think you missed the next part where I explained that this was an issue years ago that has been fixed with the architecture-based priority inversion system.

bayviewboom:
packages with modified libs should NOT be able to overwrite the stock libs. That's insane. They should be required to put their bundled versions in their OWN folder, and not touch the stock versions.

Again, override, not overwrite. Packages don't overwrite anything.

bayviewboom:
They should be required to put their bundled versions in their OWN folder, and not touch the stock versions.

That's exactly what they do.

bayviewboom:
Library Manager shouldn't touch anything in Documents folder.

If by "Documents folder" you mean the sketchbook folder (which can be in any arbitrary location, but is under Documents by default on Windows), then I agree.

bayviewboom:
Arduino is installed to Program Files, as it should be. Any libs added by Library Manager or a 3rd-party package should go into Program Files

I disagree with that.

bayviewboom:
or another system folder. Not Documents!

OK

bayviewboom:
If they are mods of standard libs, i agree-- they should be stored elsewhere-- but not in Documents! AppData seems a good location, except then it's exclusive to that user. Dumb!

Why? It's convenient for each user to be able to have their own unique set of libraries installed.

bayviewboom:
Thank you Arduino for giving us arduino hell.

I'm sure Arduino would be happy to provide you with a full refund of the purchase price of the Arduino IDE if you're not satisfied.

pert:
It is simple, to the user. That doesn't mean it should or can be simple under the hood.

No, it's not simple for the user when the user gets warnings about multiple copies, or the user can't compile. The first post in this thread is one of several related user-posts.

"I found myself descending into a hell of conflicting libraries and juggling different versions of the Arduino IDE on different computers. It was all getting very confusing and difficult to maintain."
https://www.meteocercal.info/forum/Thread-Running-the-Arduino-IDE-as-a-portable-app

They don't need to know anything about the include priority system. That would be very limiting and the users would be worse off for that restriction.

What restriction? The restriction of not having a confusing mess on their hands?

The priority system allows us to install updates of the libraries bundled with the Arduino IDE as new versions are released.

No, the updater-process does that. That has nothing to with priorities. If only one single copy of a given version were allowed, that version could be updated even without a priority system.

It allows us to write our own custom replacements for the bundled libraries.

custom replacements should be required to be stored in the same directory as the sketch, not stored in a common folder where it can cause conflicts. A priority system wouldn't be needed in that case.

They are.

They are? They why does the compiler report conflicts?

What are "all other sketches"?

A custom lib store in a sketch folder shouldn't be recognized as a conflicted lib when compiling some unrelated sketch.

If by "Documents folder" you mean the sketchbook folder (which can be in any arbitrary location, but is under Documents by default on Windows), then I agree.

Glad you agree. I meant Windows Documents folder, the official one. I just uninstalled Arduino, and it deleted all my sketches -- that's nuts. No software installer/uninstaller should be able to touch my Windows Documents folder. The uninstaller DIDN'T remove Documents\Arduino\libraries. That's even more ridiculous, considering that i checked the "remove all extra stuff" during the uninstall. .

It's convenient for each user to be able to have their own unique set of libraries installed.

There should be options for both scenarios-- user-specific or system-wide. In this day and age, most workstations are single-user.

I'm sure Arduino would be happy to provide you with a full refund of the purchase price of the Arduino IDE if you're not satisfied.

C'mon now you can't be serious. They provide a software system so people will pay for their hardware. They want people to use it, don't they? The forum has a 'suggestions' area. And please don't tell me "write your own", this tool is intended for beginners, who obviously aren't about to write their own IDE and development ecosystem.

bayviewboom:
No, it's not simple for the user when the user gets warnings about multiple copies

Why do you think the IDE displaying useful information is not simple?

bayviewboom:
or the user can't compile. The first post in this thread is one of several related user-posts.

There is no indication that the compilation error was in any way related to the IDE's include priority system.

bayviewboom:
"I found myself descending into a hell of conflicting libraries and juggling different versions of the Arduino IDE on different computers. It was all getting very confusing and difficult to maintain."
Running the Arduino IDE as a portable app

Their problem had nothing to do with the Arduino IDE's include priorities system. They would have had the same problem even if we had your desired "only one system-wide (or at least user-side) version of a lib."

bayviewboom:
What restriction?

I already explained this in my last reply.

bayviewboom:
No, the updater-process does that. That has nothing to with priorities. If only one single copy of a given version were allowed, that version could be updated even without a priority system.

That would mean writing to the IDE installation folder, which is very much not a standard thing to do, and may not even be possible depending on your security setting (Windows has special security restrictions on the Program Files folders).

bayviewboom:
custom replacements should be required to be stored in the same directory as the sketch, not stored in a common folder where it can cause conflicts. A priority system wouldn't be needed in that case.

It's already possible to do this, and that's a very nice feature. But making this a requirement would be horrible. It would mean you have to have a copy of the custom library in every sketch that uses it. That would make maintenance of the custom library a nightmare.

bayviewboom:
They are? They why does the compiler report conflicts?

I don't know because I've never seen it happen after using the Arduino IDE and providing support for other Arduino IDE users for 7 years. Have you?

bayviewboom:
A custom lib store in a sketch folder shouldn't be recognized as a conflicted lib when compiling some unrelated sketch.

Of course. But the Arduino IDE doesn't do that, so it's no problem.

Make sure you understand the difference between "sketch folder" and "sketchbook folder". The sketch folder is the folder that contains a single sketch. The sketchbook folder is the folder defined in the Arduino IDE's File > Prerences > Sketchbook location. This folder is a convenient location to store sketches, though they are not required to be stored there. But it's also the place where libraries are installed, in the libraries subfolder.

bayviewboom:
I just uninstalled Arduino, and it deleted all my sketches -- that's nuts. No software installer/uninstaller should be able to touch my Windows Documents folder. The uninstaller DIDN'T remove Documents\Arduino\libraries. That's even more ridiculous, considering that i checked the "remove all extra stuff" during the uninstall.

Since it's unrelated to the subject of this thread, let's confine discussion of that issue to the dedicated topic you created:

bayviewboom:
There should be options for both scenarios-- user-specific or system-wide. In this day and age, most workstations are single-user.

OK, but then what happens if the user wants to use a different version of the library thatn the system-wide one that's installed? I guess we'd need some sort of include priority system that allows the user's installed version to override the system-wide one. Uh oh!!

bayviewboom:
They provide a software system so people will pay for their hardware.

The IDE also supports the millions of derivative, clone, and compatible boards that Arduino doesn't make a penny off.

bayviewboom:
They want people to use it, don't they?

Sure, but you're not required to use it. There are other options if you don't like it.

bayviewboom:
The forum has a 'suggestions' area.

You're welcome to express your opinion here. I'm also welcome to do the same.

bayviewboom:
No, it's not simple for the user when the user gets warnings about multiple copies, or the user can't compile. The first post in this thread is one of several related user-posts.

It usually is simple for the user. These issues typically don't occur unless they did something to mess up their Arduino environment.
What can be difficult for users, when things do break, is to figure out how to fix their environment.

In nearly all the cases I've seen over the years where things get wonky with multiple copies of a library, the user did something to cause it.
(I say "nearly all" because I'v seen a few cases where a bad developer screwed up their github repository and it confused the IDE library manager)
The way this tends to happen is that the user did something incorrectly, like manually installing a library incorrectly, or installing the library in multiple places, or installed multiple versions of the same library or installing multiple versions in different places.

No, the updater-process does that. That has nothing to with priorities. If only one single copy of a given version were allowed, that version could be updated even without a priority system.

It is much more complex than that. There are issues related to multiple copies of the same version of a library installed (happens from the user incorrectly installing it multiple times in multiple locations), the issue of multiple copies of a library, regardless of version, being installed, and then there is the issue of how to handle different MCU architectures for a library.

A single library for multiple architectures was the original approach that the Arduino.cc team had proposed for Arduino 1.5 libraries to support multiple architectures.
i.e. a single library that supported all architectures.
This way of trying to do things does not take into consideration how multi-platform libraries would be implemented and maintained in the real-world, particularly for 3rd party Arduino libraries and cores.
There were a few of us that lobbied heavy to the Arduino developers to NOT have a single library system.
(We lost and the original 1.5 library system was a "all in one" library design)

A priority system makes more sense when trying to solve issues like how to handle libraries across many architectures.
Consider this, if you have a system that has a single library with multiple architectures, who owns it? who maintains it?
How do you get support for your new core into it?
Is it done with ifdefs, is it done with a separate tree for each architecture?
The Arduino team had decided to use separate sub directories for each architecture rather than ifdefs all within a single library.
While this sounds nice and makes the code for each architecture cleaner/simpler looking , it makes maintenance extremely difficult as now anybody that wants/needs support for their 3rd party core must get their code into the tree for the single library which is controlled by someone else.
It also means that if there is a bug, or enhancement, all the sub directories for each architecture within the library tree must be updated.

In the case of a 3rd party library, you have to go to that third party library owner, and ask to get the code added/updated to their library/tree, or fork the library to include your code, and then try to convince users to install your fork rather than the original source for the library.
This does not scale.

Trying to use a "all architectures in one library" quickly becomes completely unmanageable.

It took a while, but the Arduino.cc deveopers finally saw the light and went away from the "all architectures in one library" using subdirectories for each architecture approach and added support to what you see today where there is a priority of multiple places that the IDE will look for a library which allows 3rd party core developers to ship their own version of the library with their core.

While there are still some issues (serious issues) with the current priority system used by the IDE for library selection, it is much better than having a single "all architectures in one library" approach.
This allows a 3rd party core to ship its own version of a library that it can update/modify/test/verify independently of another cores library as well as offering the user the ability to override libraries with their own personal version.

And for the most part, most users do not ever see or run into these issues unless they have done something like improperly install libraries (like installing multiple versions a library directly from zip file obtained from a github page) or are using a machine that has multiple versions of the IDE installed on it.
Most users don't do either of these, so they don't run into some of these remaining issues.

One of the way things went south with the priority system and the library manager is that the IDE starting using the user sketchbook for its own IDE bundled library updates.
For example, it may want to update LiquidCrystal or Ethernet independently of the IDE, so it puts the updated IDE bundled library in the users sketchbook directory.
That is just plain wrong! Not only is it potentially stomping on a users personal library that he may already have there, but it can break other versions of the IDE that might be installed on the same machine since the updated library may not run on older/different versions of the IDE.

The original idea for IDE bundled library updates was to update IDE bundled libraries "in place", which is the correct thing to do; however, the reason that this was not done was that there was a concern that the user may not have the needed permissions to do this.
That was a valid concern, however the solution of switching to just using the sketchbook library area was the WRONG solution to this.
IMO, this wasn't really a problem as it could have been as simple as doing the updates in place and updating documentation to say that updating IDE bundled library requires having the same permissions that were used for the original install.

Another thing that is missing from the current IDE library priority system is using a "libraries" directory under the sketch directory.
This could be very useful. This directory location should be of the highest priority. It would allow things like overriding and specifying specific library code for an individual sketch.
It would also allow bundling and entire project, sketch and all needed libraries into a single tree.

Another thing that is missing from the IDE library manager is a way to remove sketchbook libraries.
Arduino.cc developers have resisted this feature request for years. I have no clue why.
Having this ability could help some users clean up the mess they made by installing multiple versions of libraries.
It would not be that difficult to implement, but the dev team is pretty insistent on not doing it.

Another area where the library manager could help users from getting into trouble would be to have some better handling of zip files obtained from github repositories.
The issue is that github creates "release" zip files with a name and sub directory that matches the library name and tag
"libname-tagname.zip".
If the IDE library manager is used to import the library from that zip file, you end up with a subdirectory named:
"reponame-tagname" instead of "reponame". So users that do manual installs using zip files from github end up with incorrectly installed libraries.
For example:
if a hd44780 library in a github repo name of hd44780 has a tag of 1.1.0 the github zip file will be named hd44780-1.1.0.zip and the top level directory will be called, hd44780-1.1.0
The problem with this is that this will create a sub-directory named "hd44780-1.1.0" instead of just "hd44780" in the users sketchbook area and most users don't go back and manually rename it correctly.
This is an incorrect library installation. If the user does this again in the future to update the library, they end up with multiple versions of the library incorrectly installed in their sketchbook area.
Note that this is only an issue when using zip installs with zip files from github.
If the library can be installed & updated using the automated capabilities in the IDE library manager, this will not occur.

What makes this zip installation a problem is that, as mentioned earlier, the IDE does not have a way to remove libraries.
Users may realize the trouble they have created, but do not know how to remove previously installed libraries.

IMO, adding the ability to remove libraries from the IDE, and making the IDE zip file installer smarter to know how to trim github filenames names to create proper Arduino library sub-directory names, would go a long way to help preventing much of the user "multiple" library issues I've seen.

--- bill

A very old thread has been woken up :slight_smile:

Tell you what …I simply don’t care nowadays and leave it to Arduino to pick from multiple libraries and just ignore the resulting warning.

No point in trying to understand everything under the sun ( or hood ?!) :o

For the most part, ignoring the “Multiple libraries found” message (I’m not sure it’s accurate to call it a “warning”) is the correct thing to do. It’s only when something is not working as expected that it’s worth taking a look at that message to make sure the IDE is using the library you intended it to.

I really like that the IDE provides this information, but it does cause problems with some confused beginners. This has been happening much more frequently lately because there is a bug (fixed already) in Arduino IDE 1.8.10 that causes spurious “Multiple libraries found” messages. Sometimes they will go on a wild goose chase thinking the multiple libraries is the cause of a completely unrelated error (because the message comes at the end of the output and they don’t bother to scroll the console window up to see the real error). Sometimes the message will cause them to think something is wrong when everything is fine.

I suppose a potential solution would be to print the message higher up in the output, so the compiler errors come after it. Or it could be only printed when verbose output during compilation is checked, which already happens to some extent. My problem with the latter is that, other than the “Multiple libraries found” message, the verbose output is only useful in very rare cases, so I prefer that option be left disabled to avoid cluttering up the output and causing it to exceed the forum’s 9000 character limit.

I still need help to decide how to fix the error message: ‘Multiple libraries were found for “xxxxxx.h” compilation terminated.’

Tried to add the path to “#include <xxxxxxy.h>” does not work as the IDE demands a filename in quotes or <> and does not compile with path inserted.

AND dealing with which libraries to delete!
Thanks

guardineer:
I still need help to decide how to fix the error message: ‘Multiple libraries were found for “xxxxxx.h” compilation terminated.’

Tried to add the path to “#include <xxxxxxy.h>” does not work as the IDE demands a filename in quotes or <> and does not compile with path inserted.

AND dealing with which libraries to delete!
Thanks

Please post the full output.