Pages: 1 [2] 3 4 ... 6   Go Down
Author Topic: Comments on Arduino 1.5 Specifications  (Read 13025 times)
0 Members and 1 Guest are viewing this topic.
France
Offline Offline
God Member
*****
Karma: 9
Posts: 613
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@cmaglie

Thank you for your comments.

But what about the other points?

Quote
Please note there's a third option: as for Energia, the libraries folder is under the main architecture folder.
Logged


Offline Offline
Full Member
***
Karma: 10
Posts: 162
IDE for Microsoft Visual Studio and Atmel Studio 6.1, optional USB Debugger
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@avenue33

Not sure if this helps but arduino 1.5.3 support two library structures. The new one and the old one. So for awakward cross compatible libraries we can use the old structure and #defines?

Sorry if this is not helpful
Logged

Arduino for Microsoft Visual Studio Pro and Atmel Studio 6.1 http://www.visualmicro.com
Arduino Debugger http://www.visualmicro.com/post/2012/05/05/Debug-Arduino-Overview.aspx

France
Offline Offline
God Member
*****
Karma: 9
Posts: 613
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

What I see is the Arduino framework is widening the gap with the C++ standards.

I hope the Wiring++ framework would be more more canonical and make-frendly.
Logged


Forum Administrator
Milano, Italy
Offline Offline
Sr. Member
*****
Karma: 22
Posts: 292
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

(sorry for the delay, there days are very busy for me)

@Jantje, you already answered yourself: including the following two paths

Code:
./libraries/Servo2/src
./libraries/Servo2/arch/${arch}

should solve your problem, is that a real burden?

Using a file like:

Code:
#ifdef ARDUINO_ARCH_AVR
#include ../arch/avr/Servo.cpp
#elif ARDUINO_ARCH_SAM
#include ../arch/avr/Servo.cpp
#else
#error The hardware is not supported
#endif

doesn't scale very well, every platform you're going to support need a separate #ifdef section to be added, and you should replicate this structure for every .c / .cpp file contained in your library, this means a quadratic load of maintenance work (N_of_files * N_of_platforms) just to make it compile.

Quote
3) It is not obvious from the code what is going on.
4) There is no easy way to notice a hardware is not supported.

It's written into "platforms" field of library.properties, IMHO is much more clearer than digging into the code

@avenue33

The goal is to allow to compile the same sketch (without changes) across multiple architectures so, to allow this, there should be a common header and multiple implementations (one for each architecture).

Personally I don't like to duplicate a library into many different places, because this leads to have only a subset of them updated. I've experienced this with many Arduino libraries: some contributions slightly change the API (=> the common headers) on the AVR side but doesn't care to fix things for the less popular SAM.

BTW, if you like to use the libraries folder as you do with energia, I don't see any reason to not continue to do so.
Logged

C.

France
Offline Offline
God Member
*****
Karma: 9
Posts: 613
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

It's written into "platforms" field of library.properties, IMHO is much more clearer than digging into the code

@avenue33

The goal is to allow to compile the same sketch (without changes) across multiple architectures so, to allow this, there should be a common header and multiple implementations (one for each architecture).

Thank you for taking the time to answer our questions.

How this key library.properties file is going to be maintained?

Let's take the following case:

The library.properties file initially contains
Code:
architectures=avr,sam

Now,
  • maker A releases a board based on architecture archA and updates the file accordingly
Code:
architectures=avr,sam,archA

  • maker B releases another board based on architecture archB and modifies the file to fit its needs
Code:
architectures=avr,sam,archB

Who is going to ensure the file integrity and consolidate all the platforms to obtain
Code:
architectures=avr,sam,archA,archB

We might end up with many different versions of the library.properties file!
« Last Edit: September 11, 2013, 06:22:52 am by avenue33 » Logged


Forum Administrator
Milano, Italy
Offline Offline
Sr. Member
*****
Karma: 22
Posts: 292
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

EDIT: Hopefully, Maker A and B would send their patch to Maker C (the original library author/maintainer), to include it upstream, instead of making their fork.

In this case the work needed by C is to accept the pull request coming from A and B and, eventually, solve the conflict on the "architectures" line of the library.properties.

BTW this is a process that cannot be enforced by the tool.
Logged

C.

Belgium
Offline Offline
Edison Member
*
Karma: 58
Posts: 1731
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

(sorry for the delay, there days are very busy for me)

@Jantje, you already answered yourself: including the following two paths

Code:
./libraries/Servo2/src
./libraries/Servo2/arch/${arch}

should solve your problem, is that a real burden?
It is not a real burden but it is work that moves from the compiler to the tool. Also from a user perspective removing a library is removing 2 folders.  I realize this is not a problem for the Arduino ide but it is one for other tool builders.
Please take into account what I have stated before
Quote
Most of the 1.5.X changes are great improvements but I see this library change as over-engineering because of the focus on multiplatform. Many libraries are not multiplatform. I think there are enough c++ ways to solve the multiplatform problems for the libraries that need it; without dependencies on the IDE/make.



Using a file like:

Code:
#ifdef ARDUINO_ARCH_AVR
#include ../arch/avr/Servo.cpp
#elif ARDUINO_ARCH_SAM
#include ../arch/avr/Servo.cpp
#else
#error The hardware is not supported
#endif

doesn't scale very well, every platform you're going to support need a separate #ifdef section to be added, and you should replicate this structure for every .c / .cpp file contained in your library, this means a quadratic load of maintenance work (N_of_files * N_of_platforms) just to make it compile.
I think the file scales just as good as a folder with subfolders. To be honest I don't see the problem.
If more files are needed they should be included in the main file. This because servo.cpp is part of the description and not of the implementation.

I guess that I did not explain my main objection to the current proposal clearly.
My main objection is that the source is not under the source folder. The source is actually in a subfolder of the arch folder and the declaration is in the source folder. That is at least for the new architecture dependent libraries.
The architecture independent libraries (and I feel this is the biggest bulk) will have there source under the src folder.

My second objection is that I feel this is overengineering
The setup you are proposing may be good (except for my first objection) if you have a library witch is architecture dependant and non-related people are delivering the source for each architecture.
However as you posted in your last answer in that case someone will have to take ownership to arrange for the code.
Paul who has been working on many cross architecture libraries has states he prefers to work with ifdef in the code.
I call this over engineering as you are basically trying to solve a problem (multiple independent people delivering a implementation for the same declaration for different architectures) that does not exist (as they will need to deliver to a single owner) and is only a very small fraction of the existing libraries.
Think about this situation: Paul is working on a architecture dependent library and he is doing the teensy part.
I do the avr and sam part. and you are the owner.
Paul delivers 1 file Platformdependentlib.cpp for teensy
I deliver 1 file Platformdependentlib.cpp for avr and sam.
How will you organize this? Will you copy my file to the sam and avr folder? Will you ask me to deliver 2 files?


I agree you are not directly enforcing this setup to library developers but you are enforcing this to the tool makers. This because there will be a competitor screaming the tool is not 100% compatible even though there is not 1 single library around that uses the proposed setup.

Best regards
Jantje


Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

0
Offline Offline
God Member
*****
Karma: 24
Posts: 587
Always making something...
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

EDIT: Hopefully, Maker A and B would send their patch to Maker C (the original library author/maintainer), to include it upstream, instead of making their fork.

In this case the work needed by C is to accept the pull request coming from A and B and, eventually, solve the conflict on the "architectures" line of the library.properties.

History has shown "Maker C" often abandons their library after about 6 months to a year, after they've completed whatever project motivated writing the code.  Even today, there are still many minor Arduino libraries that have "#include <WProgram.h>" for pre-1.0 Arduino.

Libraries published by companies that support a shield, breakout board or other hardware they sell are an exception, but even with those, not long after initial release there's little incentive for those 3rd parties to merge patches or pull requests from Maker A & B.  They generally will merge patches from "Maker A" who is also a commercial operation and has a sizeable installed base of boards on the market, but "Maker B" who is a hobbyist selling few if any boards will generally be ignored.

Quote
BTW this is a process that cannot be enforced by the tool.

Well, of course not.

But the tool could support a way for Maker A and B to supplement the library with extra info, in a way that won't clobber each other and any revisions from Maker C.

I'll certainly be in the Maker A or B roll regarding many libraries.  If Maker C won't accept my patches, I'll fork or patch the library as needed.  I'll try to do so in the best way your infrastructure supports.

I'm really glad you're concerned about this stuff.  1.5.x is already a huge improvement over 1.0.x.   But in designing this structure, it's pretty clear from the many years of Arduino's history that depending on library authors to actively maintain their code and merge patches from 3rd parties is not very practical.
« Last Edit: September 12, 2013, 04:58:28 pm by Paul Stoffregen » Logged

France
Offline Offline
God Member
*****
Karma: 9
Posts: 613
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Paul

Thank you for joining the conversation.

One easy way to make every maker happy and avoid central planning, is to set as many sub-folders as architectures... actually pretty much how it works today.

I really like the way Teensyduino and Energia, to name a few, organise the libraries for multiple platforms.
Logged


Belgium
Offline Offline
Edison Member
*
Karma: 58
Posts: 1731
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

I see this has been implemented in 1.5.4. The sd library uses the src folder and as such is incompatible with my plugin.
I assume our arguments have not been heard.
Can someone of the core Arduino team confirm this?
Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

Forum Administrator
Milano, Italy
Offline Offline
Sr. Member
*****
Karma: 22
Posts: 292
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset


I see this has been implemented in 1.5.4. The sd library uses the src folder and as such is incompatible with my plugin.
I assume our arguments have not been heard.
Can someone of the core Arduino team confirm this?

If your request is to remove the "arch" folder, then the answer is no
(btw, it was already done and planned right before I noticed that this discussion was going on).

The basic point is that it's confusing to have multiple versions of the same library.
Like, say, take a library that communicates with an external display, say adafruit, just to name one, makes a library that talks to it, and they also provide an optimized version for AVR that uses hardware SPI (or whatever).
If someone else optimizes the library, for example for pic32, I'd rather have them get adafruit to merge the pic32 optimization into the adafruit library, instead of making a new pic32 display library, and in this case, I feel like having separate folders makes it easier, because adafruit can just grab the arch/pic32/ folder straight from the other person and make them responsible for it as opposed to having everything in one folder or one file, which makes it harder to coordinate. Also, adafruit can improve the common part of the library by touching the code in src/*.cpp folder, providing better graphics algorithms for example, that everyone can benefit.

Using a single folder, or #ifdefs within a single file, makes it harder to provide versions of a library for multiple architectures, not easier.

Libraries published by companies that support a shield, breakout board or other hardware they sell are an exception, but even with those, not long after initial release there's little incentive for those 3rd parties to merge patches or pull requests from Maker A & B.  They generally will merge patches from "Maker A" who is also a commercial operation and has a sizeable installed base of boards on the market, but "Maker B" who is a hobbyist selling few if any boards will generally be ignored.

I repeat, this is not a tech matter, and you will face the same problem, even worst, with a single src/ folder, because it requires more work from the original maintainer to include your code.

But the tool could support a way for Maker A and B to supplement the library with extra info, in a way that won't clobber each other and any revisions from Maker C.

That's exactly the point of the arch folder. Or maybe you have something else in mind?

Also from a user perspective removing a library is removing 2 folders.

The user remove just the Servo folder that contains both Servo/arch and Servo/src (and all the others files).

It is not a real burden but it is work that moves from the compiler to the tool. I realize this is not a problem for the Arduino ide but it is one for other tool builders.

We should make things simpler for users (and library authors in this case), and this makes life harder for tool's author (both tools and Arduino IDE developers).
BTW this time is just a little more harder, I'm sure you felt much more pain when you faced the auto #inclusion of libraries, or the sketch preprocessing, no? :-)

My main objection is that the source is not under the source folder. The source is actually in a subfolder of the arch folder and the declaration is in the source folder. That is at least for the new architecture dependent libraries.
The architecture independent libraries (and I feel this is the biggest bulk) will have there source under the src folder.

You can have source code in BOTH arch/ and src/, the difference is that in src/ you will put only architecture-independent code.

For example the Ethernet lib has a common src/ with all the supported protocols, and various SPI implementation that are architecture specific.
https://github.com/arduino/Arduino/tree/ide-1.5.x/libraries/Ethernet

The Stepper library is a completely architecture independent library, and have no arch/ folders at all.
https://github.com/arduino/Arduino/tree/ide-1.5.x/libraries/Stepper

Think about this situation: Paul is working on a architecture dependent library and he is doing the teensy part.
I do the avr and sam part. and you are the owner.
Paul delivers 1 file Platformdependentlib.cpp for teensy
I deliver 1 file Platformdependentlib.cpp for avr and sam.
How will you organize this? Will you copy my file to the sam and avr folder? Will you ask me to deliver 2 files?

I would ask you to provide two file for arch/sam and arch/avr folders, instead of one full of #ifdefs. In this case Paul becomes responsible for arch/kinetis and you for arch/avr and arch/sam. When I receive updates from you, I need to just copy your file straight in their specific folder.

In any case what's the alternative you're proposing? putting all the architecture in one big file? In this case it seems more complicated to merge your patch (because it actually needs code review) and probably I'll give up.

Logged

C.

Belgium
Offline Offline
Edison Member
*
Karma: 58
Posts: 1731
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

C
thanks for the answer.
No thanks for ignoring the community (again).
Jantje
Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

0
Offline Offline
God Member
*****
Karma: 24
Posts: 587
Always making something...
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Cristian,

Overall, I believe 1.5.4 is a huge improvement over 1.0.5.

For years I've been patching/fixing lots of libraries (often fixing bugs and improving performance, not just porting to Teensy), and since releasing Teensy3 one year ago, I now patch nearly all of the ones that ship with 1.0.5.  My aggressive patching of files has worked reasonably well for only 2 reasons: #1 most other 3 party board makers don't put much work into porting/patching libraries, and #2 they tend to fork the entire IDE.

In designing this system, I understand you're facing a lot of trade-offs.  From what I know so far, not having actually worked closely with the code yet, it looks pretty good.

If I could ask you to consider just one thing as you continue to develop the software, please keep in mind many library authors abandon their project after a period of several months to a year or two.  Long-term, depending on the original author to accept and merge patches is just not realistic.

Quote
I would ask you to provide two file for arch/sam and arch/avr folders, instead of one full of #ifdefs. In this case Paul becomes responsible for arch/kinetis and you for arch/avr and arch/sam. When I receive updates from you, I need to just copy your file straight in their specific folder.

That's actually very nice, for the source code, at least from my point of view.  smiley

I'm sure it doesn't sit well with tool authors using Makefiles.  In fact, on my long-term todo list is developing a better Makefile.  It's also certainly going to need a small collection of binary helper tools to be compatible with 1.5.X.  Extra work, but I'm really ok with extra work!  I'll only want it to be stable do I don't have to redo all that extra work....

Quote
In any case what's the alternative you're proposing?

Well, one sore spot would be modifying the library.properties file.  For the code, in theory, all my platform specific stuff goes in its own location.  But do I get to create my own .properties file that augments the original?  Or do I have to modify the one and only library.properties file for that particular library?

Hopefully the library author doesn't feel merging stuff to that .properties file is too burdensome.  But what if the library is no longer maintained?  Then I need to patch its library.properties file.  If my installer has to change that file, then we're back into a similar situation as 1.0.X, where I'm likely to "install" files that destroy changes made by other 3rd parties.

Then again, this is all still pretty hypothetical.  So far, I believe I'm the only 3rd party that's patching any substantial number of libraries without forking the entire IDE.  But nonetheless, it's a non-ideal situation that could become a problem if someone else starts patching libraries.
Logged

Belgium
Offline Offline
Edison Member
*
Karma: 58
Posts: 1731
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

After thinking about how to solve this puzzle I must confess I'm out of options.
I know how I can modify the Arduino eclipse plugin so the makefile is ok. However this will not work for the indexer. As such the indexer will not know whether you are using sam or avr which reduces the added value of using a real IDE.

There are basically 2 options I still see possible.
The first is to add a lot of code to add and remove path variables when libraries get added or removed.
The second option is to have the end user do this.

For the first option I will need lots of time (I'm talking months of work here) which I can and do not want to invest.
For the second option we are talking about a support nightmare which I do not want to handle.

It is with pain in the hart that I conclude that I currently see no other option than stopping my Arduino eclipse plugin purely for this library specification.

jantje

ps The Teensy solution I designed is also incompatible with the Arduino 1.5 Specification.
Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

France
Offline Offline
God Member
*****
Karma: 9
Posts: 613
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

It is with pain in the hart that I conclude that I currently see no other option than stopping my Arduino eclipse plugin purely for this library specification.

That might be the same fate for embedXcode, as Arduino is concerned, and for the very same reasons: the new nonsensical library structure.

This should have been avoided, should the Arduino team be more open. I started this thread a long time ago, May 11, 2013, 11:53:48 am. The first answer from a member of the Arduino team only came many months later, on September 09, 2013, 11:27:43 am.

Openness means listening to users and taking their comments into account. Arduino doesn't provide a decent IDE, so the Arduino team should at least consider the Arduino plug-ins for standard IDEs like Eclipse, NetBeans, Visual Studio and Xcode.

Ultimately, the late answer and the closeness are about Arduino having no consideration and no respect to those who, like Jantje, Visual Micro and myself to name a few, develop Arduino plug-ins for full-fledged IDEs and contribute to a larger Arduino audience.
Logged


Pages: 1 [2] 3 4 ... 6   Go Up
Jump to: