Go Down

Topic: Comments on Arduino 1.5 Specifications (Read 21131 times) previous topic - next topic

Jantje

Quote
What I was trying to solicit is to  see is if everybody else is seeing the flaws
in the current .properties specification with respect to trying to specify 3rd party cores
or architectures within those 3rd party cores.

I think we all see some flaws. Not sure we all see the same.
I was also wondering whether anyone represented the ino tool.

Quote
I didn't follow that.
You can do pretty much anything to the compiler, linker, and download commands given the
flexibility offered by the platform and board files. I had to completely change things for chipKIT but the flexibility was
there for me to change all the commands around when chipKIT boards were selected without
having to change anything in the Arduino JAVA IDE code.

What I mean is that if platform.txt states
compiler.c.cmd=avr-gcc
and you selected a UNO and the boards.txt file contains
uno.compiler.c.cmd=sam-gcc
the plugin will use sam-gcc while I expect the Arduino IDE to use avr-gcc

I fully agree the boards.txt and platform.txt makes the life of a tools developer for the compilation part easier.
There has already been a discussion on the developers list that the same should be implemented for the upload.

Quote
The user doesn't have to know any of this as long the system allows a library
to specify which environments it works in.

It is visible in 1.5.2 when you select a board

Quote
In the bigger picture, 3rd party Core, Library, and tool writers are not normal users.
They are the ones that create the s/w to make it easy for the normal users.

I fully agree on this but there is to much focus on Architecture while this could solve many other problems as well.
For instance this http://forum.arduino.cc//index.php?topic=143940.msg1081073#msg1081073
There is a library for avr for tone and you have a new implementation which is better but takes more space and is still in beta.
Some people will want the old one. other will want the new one. Why not have both and let the user select the most appropriate for him/her?
In this particular case the folder name can not be the architecture as the architecture is the same for the 2.
so you will have tone/arch/stable and tone/arch/beta
the folder /arch/stable contains a property file stating the architectures it runs on and so does arch/beta.
So it can be that /arch/stable supports avr, sam, all and chipKIT but arch/beta only avr.

Quote
What I'm trying to bring up is that from a purely functional point of view
(ignore how ugly or painful it might be to actually implement), the current
.properties spec does provide enough information to specify which
cores and sub architectures within a core are supported by the library when
3rd party cores are involved or when a library is specific to a particular chip.
And without those types of capabilities,
it is impossible to provide a nice clean & simple user experience.

Technically yes but it misses on process and ownership.
For example if I created a new tone library I want to add a folder under libraries/tone/arch/ and it should work.
I don't want to be dependent on someone else to change the existing properties file (for instance because this library supports sam implementation which is not yet available). And when the folder is removed the sam implementation is removed as well.
In other words: As a provider of the library I want to be able to specify some fields in properties. The best way to do so is add a properties file which overrules (or adds to) the properties in the parent.
I don't want to be dependent on a tool (which I would need to implement in eclipse) that updates the properties file based on user import and export activity.

Best regards
Jantje
Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

Paul Stoffregen


Paul,
What is your understanding of the "architectures=" field?


I honestly do not really know at this point.

Ultimately, my top priority is making everything "just work" on Teensy.  A strong secondary goal is enhancing the Arduino API, ideally in ways compatible with all official Arduino boards, like the advanced ADC API recently discussed on the developer mail list.

I also do want to utilize this multi-platform stuff, but that's a secondary goal.  If there's any sort of trade-off, making everything "just work" for customers takes precedence.  I already apply lots of patches to the IDE.  I want to do things in the simplest, most maintainable ways, but if patching results in more work for me but a better experience for my customers, I'll always optimize for end user experience.

Regardless of what the spec says, I believe what this really ends up meaning will largely depend on how many library authors actually end up using it.  I've never been good at predicting the future, so I'm going to resist the urge to make any more predictions.  I'm mostly in "wait and see" mode.

bperrybap


What I mean is that if platform.txt states
compiler.c.cmd=avr-gcc
and you selected a UNO and the boards.txt file contains
uno.compiler.c.cmd=sam-gcc
the plugin will use sam-gcc while I expect the Arduino IDE to use avr-gcc

I guess I don't have that expectation and don't see an issue here.
The  point of having the user pick a board is to potentially do things that are board (chip) specific,
to allow a means to potentially do something differently for a given  board (chip).
If there are different chips supported within the same chip architecture,
you must be able to potentially override any default settings in the architecture (platform.txt) file.
While the above is a very contorted example, the values of the fields in the board file for
a particular board have to be able to override the field values in the platform file.
In my view, what you have described is the way it should and needs to work.

For example, maybe  I want to specify a shell wrapper to set up my tools
for source level debugging. So maybe I set up a board type that uses a shell
wrapper script that sets up link maps, DDD, AVaRICE, and my AVR dragon as part
of the build and/or upload process. I might have to override the platform.txt values to
get it to call my wrappers instead of the calling the tools directly.
(I've done this, and  did this to get chipKIT up and working on IDE 1.5.1)


Quote
The user doesn't have to know any of this as long the system allows a library
to specify which environments it works in.

It is visible in 1.5.2 when you select a board


I think perhaps you have misunderstood me.
When there is a mechanism in place that allows a library to specify which boards (chips)
it works on, the user only has to know what board he has. How it works under
the covers is no concern to him.


Quote
In the bigger picture, 3rd party Core, Library, and tool writers are not normal users.
They are the ones that create the s/w to make it easy for the normal users.

I fully agree on this but there is to much focus on Architecture while this could solve many other problems as well.
For instance this http://forum.arduino.cc//index.php?topic=143940.msg1081073#msg1081073
There is a library for avr for tone and you have a new implementation which is better but takes more space and is still in beta.
Some people will want the old one. other will want the new one. Why not have both and let the user select the most appropriate for him/her?
In this particular case the folder name can not be the architecture as the architecture is the same for the 2.
so you will have tone/arch/stable and tone/arch/beta
the folder /arch/stable contains a property file stating the architectures it runs on and so does arch/beta.
So it can be that /arch/stable supports avr, sam, all and chipKIT but arch/beta only avr.

I think the specific library (NewTone) is a poor example, as it used a different name and different API
that the library it is trying to replace and as such does not conflict with the IDE supplied tone library.
I'll assume you mean that you want to have a new/optional/experimental library that uses the same name and API
as some other library.
That said, I'm not understanding if you are wanting the ability to select which library to use
at the sketch level (sketch A used old and sketch B uses new) or whether you simply
want the ability to let the user configure/install a library than can override an existing library
for a given vendor core & architecture.
I brought up a similar scenario in my very first post of this thread.
It was scenario #2 of this response:
http://forum.arduino.cc//index.php?topic=165827.msg1437604#msg1437604
Given that the user libraries already override the IDE supplied  libraries,
the ability to support this capability should be possible if proper dependencies exist in the .properties files.
i.e. Already today, if you install a library in your user sketchbook/libraries area it will override
a library provided by the IDE when the names are the same.
So if a future build mechanism/system maintains this capability as well as properly picking the appropriate
library to support the chip on the selected board, by looking at the various properties
in the library hierarchy, then it all will "just work".
The key is to properly define the .properties file and how the library hierarchy works.
The hierarchy mechanism should use a directory structure that doesn't require 3rd parties
to have to merge their separate properties information.


Quote
What I'm trying to bring up is that from a purely functional point of view
(ignore how ugly or painful it might be to actually implement), the current
.properties spec does provide enough information to specify which
cores and sub architectures within a core are supported by the library when
3rd party cores are involved or when a library is specific to a particular chip.
And without those types of capabilities,
it is impossible to provide a nice clean & simple user experience.

Technically yes but it misses on process and ownership.
For example if I created a new tone library I want to add a folder under libraries/tone/arch/ and it should work.
I don't want to be dependent on someone else to change the existing properties file (for instance because this library supports sam implementation which is not yet available). And when the folder is removed the sam implementation is removed as well.
In other words: As a provider of the library I want to be able to specify some fields in properties. The best way to do so is add a properties file which overrules (or adds to) the properties in the parent.
I don't want to be dependent on a tool (which I would need to implement in eclipse) that updates the properties file based on user import and export activity.
Best regards
Jantje



I totally and completely agree on the need to separate out the property fields  and not require
library developers to have to coordinate and merge their properties and directory trees.
As I mentioned earlier, I believe that requiring 3rd parties to have to cooperate
so closely is unlikely to work since it doesn't scale well across different developers.
To me the current 3rd party cooperation/merging requirement
is a one of the biggest short comings of the current 1.5 library specification.

However, it isn't that simple when it comes to using processor "arch" because
what does "arch" really mean when using 3rd party cores?
Again, think 3rd party cores, not just the cores that come with
the Arduino IDE. Toss in Teensy, maniacBug1284p, and attiny cores.
This HUGE ambiguity over architecture is another point I've brought up several times.
According to the 1.5 hardware spec, which is what supposedly defines how new cores are added,
"arch" is a component of the vendor/maintainer core.
Because of this, there isn't necessarily a single core for the "avr" architecture.
So you can't get away with using just architectures names in the .properties file.
For example, while the Arduino teams supplies the Arduino/avr core,  it doesn't support all the avr chips.
Teensy will be supplying  a partial avr core, and so will maniacbug, and attiny.
A library may only work on the AVR processors that use the Teensy core.
There is no way to specify this.

So again, I believe that it comes back to properly defining the desired functionality first.
In order to do the things like you mentioned above there must first be a requirement to do it,
then there must  be way to actually specify it.
Then,  there can be discussion over implementation,
which includes the directory structure and the mechanisms to support it.
Some of these are inter-related and intertwined and will need to be considered
at that same time.
However, I think trying to jump directly to the directory tree implementation
before agreeing on the functionality much less the way to specify it, is jumping ahead
too far.

--- bill


cmaglie

#78
Oct 24, 2013, 10:43 am Last Edit: Oct 24, 2013, 10:46 am by cmaglie Reason: 1
I see a certain of confusion about what "architectures" means in the library specification, so let me clarify a bit.

Let's pick the example by bill about maniacBug1284p:

The vendor of this board should proceed this way:

1) create a 3rd party core for his board:

Code: [Select]

hardware/maniacbug/avr/platform.txt
hardware/maniacbug/avr/boards.txt
hardware/maniacbug/avr/cores/arduino/[source code for "arduino" core of maniacbug]


2) create a library that has "platforms=avr" property set.

And he's done.

How the IDE can understand that the library author wants to use the maniacbug core found here:

Code: [Select]

hardware/maniacbug/avr/cores/arduino/[source code for "arduino" core of maniacbug]


instead of the Arduino core found here?

Code: [Select]

hardware/arduino/avr/cores/arduino/[source code for "arduino" core of ARDUINO]


simple: because the user has previuosly select the board "Maniac Bug" (or whatever) from the "Tools" menu.
The definitions for this board are inside the boards.txt under maniacbug hardware folder so the IDE knows that it should use the "arduino" core inside that folder.

The concept of "vendor" is there for this reason, we can have multiple vendors that implements the same architecture!

C.

Paul Stoffregen

#79
Oct 24, 2013, 10:44 am Last Edit: Oct 24, 2013, 10:59 am by Paul Stoffregen Reason: 1
Cristian, if you're still reading, here are some questions:

All 22 libraries that ship in Arduino 1.5.4's libraries folder have "core-dependencies=arduino (>=1.5.0)".  What is my Teensyduino installer supposed to do with the core-dependencies field?  If "arduino (>=1.5.0)" means it will only work with a core named "arduino", do I need to rename my core from "teensy" to "arduino"?  Or do I modify the library.properties files to add the name of my core to the properties files?

Likewise, how am I supposed to install Teensy 3.0's non-avr architecture EEPROM library?  Suppose I name Teensy 3.0's architecture "kinetis".  It seems logical that I would add a "kinetis" directory inside libraries/EEPROM/arch and copy my architecture EEPROM.cpp there?  But what do I do about EEPROM.h?  It's currently in libraries/EEPROM/src, shared by all architectures.  My EEPROM library customizes that header, adding a constructor which I use to initialize the hardware (unlike AVR, the eeprom hardware needs to be initialized).  It looks like I'll need to overwrite the platform neutral header with my customized version.  Then, of course there's the library.properties file in EEPROM, which currently says the EEPROM library is only compatible with AVR architecture.  Will I need to modify EEPROM's library.properties, for both architectures and core-dependencies?

SoftwareSerial is another library I've ported to Teensy 3.0.  Fortunately, SoftwareSerial seems to have a header in libraries/SoftwareSerial/src that includes the architecture specific header, but it looks like I might need to patch library.properties, since both architectures and core-dependencies say it's incompatible with my hardware (even if I copy the code into a new directory to make it compatible).  Another complexity with SoftwareSerial is I'll want to patch the examples, which mention the supported pin numbers only for Arduino boards.  I'll want to add similar comments for my 3rd boards.  How am I supposed to do that?

I recall earlier you mentioned the intended path is to contact the library maintainer and get them to update their library.properties file and add the code.  Is that really what you intend for the 22 libraries that ship with Arduino?  Generally Arduino has maintained a policy of facilitating 3rd party hardware, but not shipping any code or file that directly support any non-Arduino boards.  Arduino's lengthy release cycle could also be, well, frustrating if you're the path to get 3rd party support into those library.properties files.

I did notice 1.5.4 has two old style copies of the Wire library, in hardware/arduino/avr/libraries/Wire and hardware/arduino/sam/libraries/Wire.  Maybe this the path?  If I have a library like EEPROM ported to another architecture, but it's already present with library.properties incompatible with my hardware, can I put a pre-1.5 format library in my platform/arch/libraries directory?  Will it override the same-named copy in the main libraries directory?  I was under the impression those directories were the pre-1.5.3 way which was intended to be replaced by library.properties.  Is that right?  If those are the official way for 3rd party boards to be compatible with the 22 "standard" libraries that ship with Arduino, it really ough to at least be mentioned in the library spec.

When I start supporting 1.5.X, I really do want to do it properly.  I honestly do not see what the proper way is to resolve these many issues, at least not without overwriting many library files.  Please let me know your thoughts?

cmaglie

#80
Oct 24, 2013, 01:27 pm Last Edit: Oct 24, 2013, 01:54 pm by cmaglie Reason: 1
Paul thanks for trying it, real examples helps a lot my understanding of the issues.


All 22 libraries that ship in Arduino 1.5.4's libraries folder have "core-dependencies=arduino (>=1.5.0)".  What is my Teensyduino installer supposed to do with the core-dependencies field?  If "arduino (>=1.5.0)" means it will only work with a core named "arduino", do I need to rename my core from "teensy" to "arduino"?  Or do I modify the library.properties files to add the name of my core to the properties files?


Well, actually those "*-dependency" fields are not used, and needs to be better defined, so as a temporary workardound you can leave it as is.
Theoretically, "core-dependencies" refers to the core inside your "vendor" folder.
Just to be clear, I imagine (and hope :)) you're doing something like:

Code: [Select]

hardware/pjrc/kinetis/platform.txt
hardware/pjrc/kinetis/boards.txt
hardware/pjrc/kinetis/cores/arduino/....
hardware/pjrc/kinetis/variants/.....
hardware/pjrc/avr/....same stuff here...


the "core-dependencies: arduino" means that you want to use the "arduino" folder inside the "cores" folder of whatever architecture you are using. So unless you named differently the following folder:

Code: [Select]

hardware/pjrc/kinetis/cores/arduino/....


you can leave "arduino" in the core-dependency file.

Quote

Likewise, how am I supposed to install Teensy 3.0's non-avr architecture EEPROM library?  Suppose I name Teensy 3.0's architecture "kinetis".  It seems logical that I would add a "kinetis" directory inside libraries/EEPROM/arch and copy my architecture EEPROM.cpp there?  But what do I do about EEPROM.h?  It's currently in libraries/EEPROM/src, shared by all architectures.  My EEPROM library customizes that header, adding a constructor which I use to initialize the hardware (unlike AVR, the eeprom hardware needs to be initialized).  It looks like I'll need to overwrite the platform neutral header with my customized version.  Then, of course there's the library.properties file in EEPROM, which currently says the EEPROM library is only compatible with AVR architecture.  Will I need to modify EEPROM's library.properties, for both architectures and core-dependencies?


We should update the EEPROM.h and add a constructor. It's a simple fix this time, if you send a pull request I can quickly merge it.
(it would be even better to add a begin() method to avoid the C++ constructors fiasco but... anyway...).
You should append ",kinetis" to the platforms=.... field. It's still not the intended way to work (I mean patching a library with an installer) but I see your concerns about IDE release cycles.
I wanted the platform= property but this example let me think about that. I recall that you already pointed this out, maybe on my first request for comments? Do you a have a solution for that?

Quote

SoftwareSerial is another library I've ported to Teensy 3.0.  Fortunately, SoftwareSerial seems to have a header in libraries/SoftwareSerial/src that includes the architecture specific header, but it looks like I might need to patch library.properties, since both architectures and core-dependencies say it's incompatible with my hardware (even if I copy the code into a new directory to make it compatible).


Same for EEPROM.

Quote

Another complexity with SoftwareSerial is I'll want to patch the examples, which mention the supported pin numbers only for Arduino boards.  I'll want to add similar comments for my 3rd boards.  How am I supposed to do that?


Ok, as said earlier, this is an uncovered part of the specification. Suggestions welcome.

Quote

I recall earlier you mentioned the intended path is to contact the library maintainer and get them to update their library.properties file and add the code.  Is that really what you intend for the 22 libraries that ship with Arduino?  Generally Arduino has maintained a policy of facilitating 3rd party hardware, but not shipping any code or file that directly support any non-Arduino boards.  Arduino's lengthy release cycle could also be, well, frustrating if you're the path to get 3rd party support into those library.properties files.


We need to split libraries from the Arduino IDE and put them in their own repositories, its already in the Arduino plans, but these steps, even if it seems easy, requires time to be done properly. The final goal is to push authors to collaborate, and to make, say, a good SoftwareSerial library that is well maintained with a consistent API across platforms.

Of course, this is my thought, in my ideal world. Instead I see, also from comments in this thread, that current trend is that every author starts its own fork of every library, eventually changing the API, or not following the updates that the upstream library adopt in the meantime.

I know also that is frustrating to wait months for a patch to go in. And I perfectly understand your concerns.

Quote

I did notice 1.5.4 has two old style copies of the Wire library, in hardware/arduino/avr/libraries/Wire and hardware/arduino/sam/libraries/Wire.  Maybe this the path?  If I have a library like EEPROM ported to another architecture, but it's already present with library.properties incompatible with my hardware, can I put a pre-1.5 format library in my platform/arch/libraries directory?  Will it override the same-named copy in the main libraries directory?  I was under the impression those directories were the pre-1.5.3 way which was intended to be replaced by library.properties.  Is that right?  If those are the official way for 3rd party boards to be compatible with the 22 "standard" libraries that ship with Arduino, it really ough to at least be mentioned in the library spec.


I see that the library search priority is another difficult topic.
The current IDE should work, more or less, like this:

Let's call ARCH the architecture of currently selected board, and VENDOR its vendor name.
1) The IDE discards all 1.5 libraries that haven't ARCH in the platform= property.
2) From the remaining libraries it choose the first matching library by searching these folders, in order:
   a) SKETCHBOOK/libraries
   b) ARDUINOPATH/hardware/VENDOR/ARCH/libraries
   c) ARUDINOPATH/libraries

So putting a library inside ARDUINOPATH/hardware/VENDOR/ARCH/libraries works. Again, here there is another compromise to deal with: we can reorder the path priority for example b) c) a), but this won't let the user the possibility to override vendor-provided libraries.

C.

Paul Stoffregen

#81
Oct 24, 2013, 07:18 pm Last Edit: Oct 24, 2013, 07:23 pm by Paul Stoffregen Reason: 1
Thanks for taking the time to look at these issues.

Regarding the EEPROM.h, I'm afraid I oversimplified things a bit.  I'm also using inline functions that call C code from within the class definition.  I don't actually have any C++ code for EEPROM on Teensy 3.0, except in that header.  The rest is all C-only, for a number of other reasons I won't mention, for the sake of keeping this reply shorter.

I really do appreciate your offer to merge a pull request quickly.  But with 1.5.5 only days from release, I would really prefer to target 1.5.6 and allow more time to carefully consider these complex issues.  The last thing I want to do is hastily prepare a pull request and cause breakage in an Arduino release, even if it is labeled beta.

It's tempting to write more regarding the parts of this spec that give my the biggest cause for concern, "core-dependencies=" and "architecture=", but at this moment I'd really prefer to think about these a while longer before more comments.  Likewise for customizing the examples.

The library search order is one place where I'd like to make a proposal.

Quote

The current IDE should work, more or less, like this:

Let's call ARCH the architecture of currently selected board, and VENDOR its vendor name.
1) The IDE discards all 1.5 libraries that haven't ARCH in the platform= property.
2) From the remaining libraries it choose the first matching library by searching these folders, in order:
   a) SKETCHBOOK/libraries
   b) ARDUINOPATH/hardware/VENDOR/ARCH/libraries
   c) ARUDINOPATH/libraries

So putting a library inside ARDUINOPATH/hardware/VENDOR/ARCH/libraries works. Again, here there is another compromise to deal with: we can reorder the path priority for example b) c) a), but this won't let the user the possibility to override vendor-provided libraries.


I agree, a) b) c) is the best search order.

However, I would like to propose a smart strategy for the a) to b) transition.  If the user installs a library in their sketchbook, perhaps with "architecture=avr" or any other library.properties line that doesn't match the board they're selected, then b) and c) should be tried to find a better match.  Libraries found in b) should always match, and c) is the last location, so really the smart decisions are only truly needed at step a).

bperrybap

Cristian,
You did answer one thing that I thought was probably the case:
The "architecture" names are global across all vendors.

As far as my question goes about how a 3rd party library can specify which core to use
I think you have missed part of the issue.
What I was questioning was more how does a library specify that it
only works with certain specific cores?
For example, suppose the library only works on a specific vendors core
and none of the others - even if they are the same architecture.
i.e. maybe something only works on the pjrc teensy avr core.

Then there is the case where a library only works on a certain chip.
Maybe it only works on a board that has a 32u4 on it
or maybe it only works on board with a m168 or m328.
Libraries only working on specific chips is quite common.
This common real-world scenario can't be handled by the current .properties spec.

What I'm saying is that I wasn't concerned that the spec could get
a library to work when the user selected a board that supported it even when it was
on a 3rd party core or 3rd party board.
It was more that the there is not enough information to prevent the user from trying to use
it on boards that can not support it.
It was my assumption that this was a goal of the .properties stuff in the first place.
To provide enough information to allow the IDE to filter out and not show
libraries and examples that will not work on the board the user has selected.
And if we going to allow the user to pick libraries and examples that won't work
on and/or error off during compilation for the selected board,
we are missing out on what could be done because that is essentially what we have today.

And then what about sketches?
Sometimes users go off and download sketches.
Currently there is no way for a sketch author to specify which cores or libraries may
be required other than with comments in the code.
We see many users posting questions in the forums asking why the sketch or example
that they download isn't compiling because of missing libraries or it not working
on the board that they have.


I guess what bothers me is that the current spec looks like it is only capable of
solving some of the issues while ignoring several current real world issues,  and I don't really like partial solutions.


With respect to "cores" and core-dependencies,
I think I now see the problem.
The Arduino team point of view seems to be that the environment as fairly consistent
for a given release.
And for them this is case because they ship a full blob that is all done at the same time and is all the same
level/version. The Arduino team really doesn't have to deal with past releases in their libraries.
However, once you enter into the 3rd party world, this is no longer the case.
In the 3rd party world, a core or library is done and while the Arduino IDE and its cores advance
the 3rd party library and core may not be updated right away or at all.
Also, a 3rd party library and core cannot always depend on only using the latest Arduino release
the way the Arduino team can with its libraries.
Many 3rd party libraries today are capable of running on pre 1.x as well as 1.x
and even 1.5 (by using appropriate ifdefs).
And this is why backward compatibility is so important to the 3rd party world,
whereas to the Arduino team they have no need for backward compatibility
because they essentially re-do everything all at once on each of their releases
and get to leave old releases behind.
This is why the 1.0 library change was no big deal to them, while it was a HUGE deal
the 3rd party developers.

So back to why this matters for the "core-dependencies".
After seeing the response to Paul's question it appears that
there really is no vendor/maintainer core dependency in the spec at all.
The "core-dependencies" is actually a version of a architecture core within all the vendors cores.
There appears to be an assumption that all the cores of a given architecture will/must be at
the same version.

I had asked about  about what the "core-dependencies" really meant in previous posts.
My assumption had been that it referred to the vendor/maintainer to allow the library
to specify which vendor's core it worked on.
But now it appears that this is really a revision of the "arduino" core within all the architectures.
Note: The naming seems VERY confusing in that there is a "arduino" vendor
and subdirectory in the hardware directory and now we have another "arduino" name
that means something completely different.
What really confuses me is that the library spec says:
Quote
core-dependecies - the core on which this library works on (multiple cores can be specified)

So my assumption was "cores" like Teensy vs ManiacBug vs Arduino etc...
But this seems to not be the case.
It appears that the "core-dependencies" values are referring to the revision
of an architecture core.

While I can see the use for the ability of the library to ensure that a particular "arduino" core API
is supported within a given architecture core, my concern is that it appears to be tracking the
IDE revisions because that's how thins are released for the Arduino team code.
This concerns be greatly.
It also does not appear to allow using or requiring different versions for the different architectures.

Consider this case:
Down the road the "arduino" API advances and you have a situation
with multiple maintainers for a library.
For example,
the latest "arduino" API is now 1.5.9
The library previously supported an avr architecture and works on any API back to 1.5.0
Now support is being added for sam but the sam developer decided to use features
from 1.5.9
So now the sam code requires >=1.5.9 but the avr code can run on >=1.5.0
There does not appear to be a way to specify that.

This is why earlier I brought up the possibility of specifying the core version
on the architecture= values.
It allows the library to run with mismatched versions on different architectures
as well

So you could have:
architectures = vendor/architecture/core (version)

This would support most of the capabilities I've been talking about
including the ability to specify individual vendors cores.
(It wouldn't handle the library only working a specify chip).

So for the above example:
architectures=*/avr/arduino (>=1.5.0), */sam/arduno(>=1.5.9)

which is any vendor with "avr"  architecture  "arduino" core and then
the desired/needed revisions of that core.

For a library that only works on Pauls teensy avr core:

architectures=pjrc/avr/arduino

For a library that only works on maniacbug
architectures=maniacbug1284p/avr/arduino

For a library that works on any "arduino" core >= 1.5
architectures: */*/arduino (>= 1.5.0)
(any vendor, any architecture, but requires arduino core >= 1.5.0)

To me this seems to make more sense than the current
core-dependencies since I simply can't resolve how that really
works once you have more than one version and multiple architectures.

etc...

or perhaps some sort of hierarchical structure like what is in
the platform and board files to allow the specific architecture of
board preferences override the more general settings.


For sketches, I think it would seem logical to use the same
type of .properties file.
The sketch would have its library dependencies,
and those libraries may have additional dependencies
to form the final dependencies for the sketch.



All that said, I think that there needs to be more thought to come
up with a way that allows a library to specify that it only works on
a specific chip.

Perhaps using the property values like what is in the board and platform
files is the answer.

--- bill





Paul Stoffregen

#83
Oct 24, 2013, 10:08 pm Last Edit: Oct 24, 2013, 10:10 pm by Paul Stoffregen Reason: 1
Bill, precise fine-grain compatibility specs would be excellent on a technical level, but would they also work well on a human level?

A scenario that concerns me looks something like this:

1: User A writes code to interface some sensor to Arduino Uno.
2: Forum users B, C, & D request the code, so he packages it into a library and uses a library.properties for Uno.  User A posts the code to a forum topic.
3: Leonardo users can't use the library, even though it would work, only because library.properties is too narrowly specific.
4: User C with a Leonardo adds the specs for Leonardo, posts an updated copy later on the same forum topic.
5: User B has an Arduino Yun, which could run the library, except user C only owns a Leonardo board, and didn't add/test support for Yun.
6: Several months later, Arduino Tre is released.  The library could run on Tre, but can't, only because the spec is again too narrow.
7: Later, someone wants to run it on Teensy 2.0.  I make a modification for Teensy 2.0 and publish a copy in my Teensyduino installer.
8: Yet more users want to run the library on Arduino Micro, Sparkfun's clones, or other derivitives.  Maybe they find the original Uno-only code, or the 2nd port, or perhaps my version.

I could go on and on with this list.....

However, my biggest concern is how code gets ported and adapted to different boards.  Real users simply try using a library on an unsupported board.  They get errors and ask for help on the forum.  Hopefully they post the actual error messages.  The "copy error" button in 1.5.X is intended to gently guide users to post actual errors.  Expert forum users see the error and suggest fixes.  Sometimes the original author updates their library.  Sometimes abandoned libraries get picked up by others (really, I believe the most beautiful thing about open source).

My point is the "just try it and see what happens" approach, messy and error-prone, is a pretty important mechanism by which the Arduino community collaborates.  Many people who know a lot more about human interaction than me have written about the importance of empowering people to try things and accept/learn from setbacks and failures.

Yes, some libraries (notably ones you've written) are done with an expert approach, where one developer pours tremendous effort into cross-platform compatibility and verification.  But most libraries aren't developed that way.

An air-tight system that prevents users from trying to compile code on specific boards the author or maintainer hasn't certified as compatible might spare users from compile errors, but it might also have a pretty negative overall impact on community collaboration and sharing.


bperrybap

Paul,
It isn't possible to have it both ways.
Either the system allows specifying which cores/architectures/boards/chips and other libraries
it works on or it doesn't.
I'm ok with either way.
The problem is having a half way in the middle type of approach which is the way
I view the current .properties spec.

I guess it comes down to what is the point/purpose of all
this .properties stuff?
It doesn't seem to be obvious from looking at the spec or looking
at the behavior of the 1.5.4 IDE.

My assumption was that it was to ensure that all needed resources existed
to support a library. Those resources include, other libraries, cores, and hardware.

If it isn't to prevent a sketch/library from being compiled on non supported h/w
and s/w then I think there are MUCH simpler solutions to this library properties stuff.


--- bill

Paul Stoffregen

I really wish I would have put more thought into this when I first saw it several months ago.

The more I think about it now, the more concerned I feel it may create artificial barriers to humans collaborating.

cmaglie

Guys,

everyone of you made great points here.
I would ask you to continue the discussion on the developers mailing list, that is where these discussions should take place.

Please give a concise and clear recap of your concerns, and I'm sure that we can come up to a great library specification.
C.

avenue33

#87
May 19, 2014, 01:19 pm Last Edit: May 20, 2014, 03:13 pm by avenue33 Reason: 1

Guys,

everyone of you made great points here.
I would ask you to continue the discussion on the developers mailing list, that is where these discussions should take place.

Please give a concise and clear recap of your concerns, and I'm sure that we can come up to a great library specification.


Sorry, too late. I've switched for the LaunchPad platform from Texas Instruments: cheaper, better, stronger.

There's the same Processing-based IDE called Energia which is compatible with the Wiring / Arduino framework. So I can reuse all my projects with no or minimal adaptation. Moreover, Energia manages the MSP430 and TM4C platforms in a very clean way.

The LaunchPad Tiva C TM4C123 is a real bargain at US$13 with 256 KB flash, 32 KB RAM, 80 MHz ARM M4 with FPU, 40 pins and an integrated programmer/debugger. The Connected LaunchPad Tiva C Series TM4C129 goes even further: for US$20, 1 MB of flash, 256 KB of RAM, 120 MHz ARM M4 with FPU, 98 pins, built-in Ethernet and an integrated programmer/debugger.

There's the same vibrant hobbyists community (43oh and Stellarisiti forums) but also professional support at the E2E engineer-to-engineer forum hosted by Texas Instruments.

Of course, I'm using embedXcode :)

Go Up