Comments on Arduino 1.5 Specifications

avenue33:
I'd appreciate some feedback from the Arduino team.

Arduino team, are you there?

I like the idea that the 1.5.1 had ( not used later ) where it has a global 'libraries' folder, and then an architecture specific 'libraries' folder. This approach is far more standard, as you could simply define a bunch of macros specifying the Due's existence, allowing me to simply use a ifndef to exclude/include specifics.

Or I could place my entire library in an AVR specifics folder if it was not Due compatible.

Arduino needs to move closer to C++, rather than try and hide it behind the scenes. All it does for newbies is add to the confusion as they do not realise they are using C++, and parts of the standard are not applicable because of Arduino's hidden background changes and inconsistent use of file structures.

pYro_65:
Arduino needs to move closer to C++, rather than try and hide it behind the scenes. All it does for newbies is add to the confusion as they do not realise they are using C++, and parts of the standard are not applicable because of Arduino's hidden background changes and inconsistent use of file structures.

I fully agree with Arduino needing to go closer to C++. That's the very paradigm behind embedXcode, the template I've developed for Xcode: plain C++ code and makefiles.

Bump!

Arduino team, are you listening?

very unlikely
its August, Italy is closed,

Since May?

Hi,

If you want to go with the classic C++ way (a big chain of #ifdefs), you're allowed to do so just ignore the arch/ folder.
This is not explicitly stated in the documentation, maybe its worth adding it, I don't know.

When I first implemented the new library format I've checked its functionality with two example of the Servo library:

  1. the first example uses the classic chain of ifdefs:
    LibTest/libraries/Servo at master · cmaglie/LibTest · GitHub

  2. the second uses the arch/* folders way:
    LibTest/libraries/Servo2 at master · cmaglie/LibTest · GitHub

Both can be used, I could say that its a matter of taste, personally my favorite is the method 2).

Also note the with the library.properties you can filter the library from being displayed/compiled on architectures that are not supported by the library itself.

Thank you for the explanations.

My only concerns is the header is on one folder while the code is on another.

Not easy to deal with using plain makefile!

Another problem is, the shared header is compulsory for any new architectures.

  • What happens if a new architecture requires a modification of the header?
  • Who is going to maintain this repository?

My fear is, this structure looks like pretty much a walled garden...

Please note there's a third option: as for Energia, the libraries folder is under the main architecture folder.

So there's a hardware/lm4f/libraries/servo folder and a hardware/msp430/libraries/servo folder.

@avenue33

avenue33:
My only concerns is the header is on one folder while the code is on another.

Not easy to deal with using plain makefile!

why not?

if a new architecture needs a different header, this means that the header itself is architecture dependent and should be placed in one arch/xxx folder or, alternatively, the architecture-dependent piece of code needs to be placed around #ifdefs.

C

cmaglie:
@avenue33

avenue33:
My only concerns is the header is on one folder while the code is on another.

Not easy to deal with using plain makefile!

why not?

I have the same problem with the eclipse plugin that is also based on (automatically generated) makefiles.
Basically the code included in the project is "the root folder and all subfolders"
Having method 1 as you describe makes I have 1 folder for each library.
For method 2 I have to do something smart because I do not need all subfolders.
In your servo example for method 1 I only need the folder
./libraries/Servo/src ( Great I don't even need to exclude examples)
To be added to the include and source path.
In method 2 I need to include
./libraries/Servo2/src
To the include path (and probably also to the source path)
and
./libraries/arch/${arch}
to the include and source path.
I think this has following disadvantages:

  1. It is more work for the tool (while C/C++ has already fixed this)
  2. It breaks upwards compatibility for tools.
  3. It is not obvious from the code what is going on.
  4. There is no easy way to notice a hardware is not supported.

I was wondering: have you considered a c/cpp file like this?

#ifdef ARDUINO_ARCH_AVR
#include ../arch/avr/Servo.cpp
#elif ARDUINO_ARCH_SAM
#include ../arch/avr/Servo.cpp
#else
#error The hardware is not supported
#endif

Best regards
Jantje

@cmaglie

Thank you for your comments.

But what about the other points?

Please note there's a third option: as for Energia, the libraries folder is under the main architecture folder.

@avenue33

Not sure if this helps but arduino 1.5.3 support two library structures. The new one and the old one. So for awakward cross compatible libraries we can use the old structure and #defines?

Sorry if this is not helpful

What I see is the Arduino framework is widening the gap with the C++ standards.

I hope the Wiring++ framework would be more more canonical and make-frendly.

(sorry for the delay, there days are very busy for me)

@Jantje, you already answered yourself: including the following two paths

./libraries/Servo2/src
./libraries/Servo2/arch/${arch}

should solve your problem, is that a real burden?

Using a file like:

#ifdef ARDUINO_ARCH_AVR
#include ../arch/avr/Servo.cpp
#elif ARDUINO_ARCH_SAM
#include ../arch/avr/Servo.cpp
#else
#error The hardware is not supported
#endif

doesn't scale very well, every platform you're going to support need a separate #ifdef section to be added, and you should replicate this structure for every .c / .cpp file contained in your library, this means a quadratic load of maintenance work (N_of_files * N_of_platforms) just to make it compile.

  1. It is not obvious from the code what is going on.
  2. There is no easy way to notice a hardware is not supported.

It's written into "platforms" field of library.properties, IMHO is much more clearer than digging into the code

@avenue33

The goal is to allow to compile the same sketch (without changes) across multiple architectures so, to allow this, there should be a common header and multiple implementations (one for each architecture).

Personally I don't like to duplicate a library into many different places, because this leads to have only a subset of them updated. I've experienced this with many Arduino libraries: some contributions slightly change the API (=> the common headers) on the AVR side but doesn't care to fix things for the less popular SAM.

BTW, if you like to use the libraries folder as you do with energia, I don't see any reason to not continue to do so.

cmaglie:
It's written into "platforms" field of library.properties, IMHO is much more clearer than digging into the code

@avenue33

The goal is to allow to compile the same sketch (without changes) across multiple architectures so, to allow this, there should be a common header and multiple implementations (one for each architecture).

Thank you for taking the time to answer our questions.

How this key library.properties file is going to be maintained?

Let's take the following case:

The library.properties file initially contains

architectures=avr,sam

Now,

  • maker A releases a board based on architecture archA and updates the file accordingly
architectures=avr,sam,archA
  • maker B releases another board based on architecture archB and modifies the file to fit its needs
architectures=avr,sam,archB

Who is going to ensure the file integrity and consolidate all the platforms to obtain

architectures=avr,sam,archA,archB

We might end up with many different versions of the library.properties file!

EDIT: Hopefully, Maker A and B would send their patch to Maker C (the original library author/maintainer), to include it upstream, instead of making their fork.

In this case the work needed by C is to accept the pull request coming from A and B and, eventually, solve the conflict on the "architectures" line of the library.properties.

BTW this is a process that cannot be enforced by the tool.

cmaglie:
(sorry for the delay, there days are very busy for me)

@Jantje, you already answered yourself: including the following two paths

./libraries/Servo2/src

./libraries/Servo2/arch/${arch}




should solve your problem, is that a real burden?

It is not a real burden but it is work that moves from the compiler to the tool. Also from a user perspective removing a library is removing 2 folders. I realize this is not a problem for the Arduino ide but it is one for other tool builders.
Please take into account what I have stated before

Most of the 1.5.X changes are great improvements but I see this library change as over-engineering because of the focus on multiplatform. Many libraries are not multiplatform. I think there are enough c++ ways to solve the multiplatform problems for the libraries that need it; without dependencies on the IDE/make.

cmaglie:
Using a file like:

#ifdef ARDUINO_ARCH_AVR

#include ../arch/avr/Servo.cpp
#elif ARDUINO_ARCH_SAM
#include ../arch/avr/Servo.cpp
#else
#error The hardware is not supported
#endif




doesn't scale very well, every platform you're going to support need a separate #ifdef section to be added, and you should replicate this structure for every .c / .cpp file contained in your library, this means a quadratic load of maintenance work (N_of_files * N_of_platforms) just to make it compile.

I think the file scales just as good as a folder with subfolders. To be honest I don't see the problem.
If more files are needed they should be included in the main file. This because servo.cpp is part of the description and not of the implementation.

I guess that I did not explain my main objection to the current proposal clearly.
My main objection is that the source is not under the source folder. The source is actually in a subfolder of the arch folder and the declaration is in the source folder. That is at least for the new architecture dependent libraries.
The architecture independent libraries (and I feel this is the biggest bulk) will have there source under the src folder.

My second objection is that I feel this is overengineering
The setup you are proposing may be good (except for my first objection) if you have a library witch is architecture dependant and non-related people are delivering the source for each architecture.
However as you posted in your last answer in that case someone will have to take ownership to arrange for the code.
Paul who has been working on many cross architecture libraries has states he prefers to work with ifdef in the code.
I call this over engineering as you are basically trying to solve a problem (multiple independent people delivering a implementation for the same declaration for different architectures) that does not exist (as they will need to deliver to a single owner) and is only a very small fraction of the existing libraries.
Think about this situation: Paul is working on a architecture dependent library and he is doing the teensy part.
I do the avr and sam part. and you are the owner.
Paul delivers 1 file Platformdependentlib.cpp for teensy
I deliver 1 file Platformdependentlib.cpp for avr and sam.
How will you organize this? Will you copy my file to the sam and avr folder? Will you ask me to deliver 2 files?

I agree you are not directly enforcing this setup to library developers but you are enforcing this to the tool makers. This because there will be a competitor screaming the tool is not 100% compatible even though there is not 1 single library around that uses the proposed setup.

Best regards
Jantje

cmaglie:
EDIT: Hopefully, Maker A and B would send their patch to Maker C (the original library author/maintainer), to include it upstream, instead of making their fork.

In this case the work needed by C is to accept the pull request coming from A and B and, eventually, solve the conflict on the "architectures" line of the library.properties.

History has shown "Maker C" often abandons their library after about 6 months to a year, after they've completed whatever project motivated writing the code. Even today, there are still many minor Arduino libraries that have "#include <WProgram.h>" for pre-1.0 Arduino.

Libraries published by companies that support a shield, breakout board or other hardware they sell are an exception, but even with those, not long after initial release there's little incentive for those 3rd parties to merge patches or pull requests from Maker A & B. They generally will merge patches from "Maker A" who is also a commercial operation and has a sizeable installed base of boards on the market, but "Maker B" who is a hobbyist selling few if any boards will generally be ignored.

BTW this is a process that cannot be enforced by the tool.

Well, of course not.

But the tool could support a way for Maker A and B to supplement the library with extra info, in a way that won't clobber each other and any revisions from Maker C.

I'll certainly be in the Maker A or B roll regarding many libraries. If Maker C won't accept my patches, I'll fork or patch the library as needed. I'll try to do so in the best way your infrastructure supports.

I'm really glad you're concerned about this stuff. 1.5.x is already a huge improvement over 1.0.x. But in designing this structure, it's pretty clear from the many years of Arduino's history that depending on library authors to actively maintain their code and merge patches from 3rd parties is not very practical.

Paul

Thank you for joining the conversation.

One easy way to make every maker happy and avoid central planning, is to set as many sub-folders as architectures... actually pretty much how it works today.

I really like the way Teensyduino and Energia, to name a few, organise the libraries for multiple platforms.