splitting a library

My dcf77 library (GitHub - udoklein/dcf77: Noise resilient DCF77 decoder library for Arduino) got recently forked by user aido (GitHub - aido/RadioClock: Noise resilent radio clock decoder library for Arduino). His goal is to implement a msf decoder for it. Obviously once two decoders are available it is more than reasonable to split the whole thing up into several smaller libraries. However it seems that the Arduino IDE does not deal well with nested libraries. Or I am just missing how to get this going.

Has anyone any hints on how to to establish nested libaries for Arduino without requiring the main sketch to know the inner libraries?

I just Googled "Splitting Arduino libraries" and got about 240,000 hits. Surely one of those will help you.

The problem is that the IDE only supports 1 library - Core.
The other "libraries" are not built and treated as real libraries.
There are ways to do this which would allow any library to call/use any other library.
The issue is the build methodology that the IDE uses.
It does lots of "magic" for the user and part of that magic is trying to determine which
libraries the sketch uses. It does not understand or take into consideration that
other components might want to use a library so there is no consideration for which
libraries the final linked uses or needs.
This is what precludes "libraries" from using other "libraries".
A better s/w build methodology would have built all the libraries as libraries
(not just the ones you "think" are being used)
then added all the "libraries" directories to the include path
rather than adding an include path for each library.
That would allow sketches or libraries to use includes like:
#include <LiquidCrystal/LiquidCrystal.h>
#include <SD/SD.h>

And then linked the sketch image against all the libraries.
This would not only allow libraries to call/use other libraries
but it would have eliminated all the goofy parsing and searching
that the IDE has to do to try and determine which libraries are being used.
(let the linker figure it out)
It would also eliminate all the ambiguity of header files which removes
the current issue of header file name collisions.
But alas, the IDE boys didn't do things that way.

Since the build methodology in the IDE is hard coded in code and can't be changed
without changing the code itself,
the only real way to support real libraries with Arduino is to use makefiles where you can do anything you want.
Dump the IDE and use real makefiles.

--- bill

I have dumped the IDE as well as make long ago in favour of scons. This is not my issue. The issue is that others who stick to this braindead IDE (>90% of the Arduino users) will not be able to use my library without additional instructions. Till then I have to provide my library as a flat file. Although this seems to solve the issue it makes me cringe. This is what I try to solve. Seems there is no progress here for years :frowning:

I'm guessing that it is higher than 90% of the users use the IDE.
No progress in changing the build methodology. If anything the current
methodology has become further entrenched when Paul's additions to
make it aware of dates/times to prevent needless re-compiles and some
new code which has rules for how to handle header file name collisions.
It is a bit ironic that the very early IDE built makefiles and then called make.
If only it stayed on that path....

As an alternative kludge for using the IDE, you can drop multiple libraries into
the same library directory.

This works because the IDE does not care where a header file lives.
So for example, you could bundle up multiple Arduino "libraries" and put them
all in a single library directory.

library foo is independent but foobar depends on foo.

library foo, and foobar could both be put into the "foo" directory
and then shipped to the user as what appears to be a single library.

If the user sketch references any header file in that directory, the IDE will compile
all the modules in root of the directory and will add that directory to the include path.
It will then link against the compiled modules in the library.

This allows the sketch to use either foo or foobar and when foobar is included
by the sketch if any foo headers are needed by the foobar headers,
the foo headers will also be found since they reside in the same directory as foobar.

So if you want to create multiple sub libraries from your library you can as long
as you put all them in the same library directory and the sketch references at
least one of the headers in the directory.

It is a horrible hack/kludge but it is about all you can do given the way the
IDE is coded.

However it does allow you to have and ship a more complex collection
of interdependent libraries as a single bundle, which does have some advantages
for less technical users.

The only catch is that all the modules in the library directory will be built and linked
against vs just the modules that are used.
If you are careful with your coding the unused code will removed by the linker
so the final result is that you only get the code that is used.

While this kludge can work great for a collection of sublibraries that don't need
other external libraries,
it doesn't work when referencing other arduino libraries like say Wire.
In that case the sketch will still have to include a header from that library to
tell the dumb IDE that the library is being used to get the library directory to be
added to the include path and to build and link the library code.

The tricky part is handling interrupts. If you have code that uses ISRs,
You have to play some games with some weak attributes to ensure that the ISRs don't get linked in and then drag in lots of unused code.
This is because the way the compiler tools currently work, ISR code is always dragged in.

--- bill

Why do you need to split any library. You can continue to develop your library for your special elite, while the user aido can complete a library that is useful to the rest of us plebs. Nobody is going to need both.

True. But the instructions consist of something like this...

To use the DCF library you will have to add these lines to the top of your sketch...

#include <dcf_core.h>
#include <dcf_parser.h>

While very distasteful the actual cost is meager.

It also implies that all the libraries go into folders at the same level. I would say this is very distasteful.

Unfortunately I can not think of any better option. So I will probably have to bite the bullet :frowning:

But thanks for pointing this one out anyway.

(Thank you for the SCons tip.)

The latest IDE has some support for source files in subdirectories of library directories. I can't think of how that will help but I thought it worth mentioning in case you can think of something.

Yep. IDEs suck!
All library directories being at the same level seems like a very minor issue.
The bigger issue is not properly supporting libraries to begin with.

You have a few options. Here are 4:

  1. put the multiple "libraries" in a single directory and ship as a single "library" (what I described above)
    This allows the user to install a single library where all the components are in sync.
    The individual libraries will be able to find each others headers and will link against each other.

  2. break the library apart into separate libraries and have the sketch include the necessary headers to trick the IDE to pulling in all the libraries.
    This requires the user install multiple libraries.
    Other libraries are doing this kind of thing and it works well, if you can get
    the users to install all the needed libraries.

  3. keep it a single library but use separate classes for your decoders.
    Other libraries are doing this kind of thing and it works fairly well.
    This allows the user to install a single library.

  4. keep it a single library but put other library code down under IDE supported
    sub directories.
    Other libraries like the Wire library are doing this.

Arduino is a lot like Windows.
It can be made to look easy and pretty to the end user
but under the hood it is gross and a total mess.

There is no good answer given the state of the IDE.
You have to prioritize a few things and that will tend to push
you in a given direction.

One thing that I would keep as a priority is that the library should
be easy to install for the end user and that means that the IDE GUI
should be able to install it.
Many libraries out there fail to create their zip images and directory structures
in a way that allows them to be installed by the IDE.
To me this is a total fail.

--- bill