Arduino Library downloader

Hi everyone.

I don't know if something like this already exists or if this even works for everyone.

I am using the Arduino software and often i just want to know if there is already a library for the thing i want to do,
if there is a new version of a library or just to download and install a library easily.

i made an application in C++ and wxWidgets to do exactly this.
It is using currently using 7zip for extracting the libs after download.

It was my first try in coding with wxWidgets and Gui development in C++ so i hope it is not too bad. :wink:
I only tested it on my windows machine and compiled it there, but i tried to only use things that are multiplatform compatible.

I put a quick and small webpage together here:
http://ardulibrary.code-i.de/

The Sourcecode is in my SVN repository accessible with the username "gast":
http://svn.code-i.de/ardulibrary/

a small Screenshot:

there are not all librarys in the list at the moment but it does not mean i like this more than the other libs.
The list is of course maintained by hand.

a doubleclick on a library sends you to the homepage of that lib.

A good initiative! But, I would like to see this functionality as part of the official IDE. Maybe this will start a butterfly effect that results in just that.

Anyways, feel free to add any of my libraries:P
http://www.arduino.cc/playground/Profiles/AlphaBeta

nice. i have added your libs to the xml.
(it is easy if every lib have the same directory structure inside the zip-file)
The only bad is that you do not have a description for every lib.

Of course it would be nice if this feature would be added to the IDE.
Currently missing is a lib-uninstall feature in the application. Will look into this if i have time.

Nice! Even nicer would be a web-based interface that simply lists all third-party libraries and allows people to contribute, like shieldlist.org.

Just a suggestion!

--
The Rugged Motor Driver: two H-bridges, more power than an L298, fully protected

Well, that is simple. :slight_smile:

A form for entering in an URL for a zip, and a field for version, and one for a description.

Then process and store in an SQL database.

Might add a 'report bug' (and then the form would need a contact mail) and maybe a vote up/down feature.

I'll be in on it, contact me if you want to set something up.

Downloader stays as it is, but downloads the XML from the site.

Cool idea, but should someone ask the Team if this is an effort best made part of the Arduino project?

This kind of third-party repository interface has been around for many years and in several different communities. I'm wondering if there's even software already to manage the infrastructure.

For example, for Python:

For Perl:

http://www.cpan.org

For TeX:

http://www.ctan.org

In hindsight it's amazing that something like this doesn't already exist for Arduino! I guess you can say the Wiki/Playground is there but it's time for The Next Generation.

--
The Gadget Shield: accelerometer, RGB LED, IR transmit/receive, light sensor, potentiometers, pushbuttons

Completely agree!
We should get a centralized third-party repository up and running soon.

[edit]Might rewrite the client for Java? Easier cross-platform and maintainable for the Arduino developers (as the IDE is java). Sorry to OP is this is the stupidest idea ever presented.[/edit]

thanks for your kind responses.

I already thought about a posibility to post librarys.
And i would also host it on my server. I can configure it so it can also mirror it on your server.

the application might be able to load an xml from a repository like that already.

But some things need to be looked at if we do this.

At the moment everyone is compressing his own lib differently with different folder structures and different manual steps to install.

when we make a repository like that. Every library.zip should have the same structure and the structure should be checked by the upload script.

I do not know if i am allowed to host every library or what to do with librarys where the creator does not upload them on theyr own or the author is not longer aviable.

the cmd part in the xml should be removed so no one can post a malicious command in the xml. (for this every lib.zip needs the same structure to make it easier)

Doesnt need every lib checked before publishing in a repository like this?

as you see. I think the hardest part might be to get every lib-structure to be the same.
Most of them look the same but not all.

//Edit: yes. Sorry that this is not java but i can not write in java.
It is already cross platform except the 7zip exe i used.
And i know this part needs a rewrite.

I think we need to model the 'structure' after how the official team does it, and that is to zip the archive which the files resides in. That is, so one extracts within the libraries archive (this way one does not have to create a new folder before extracting).
Reference: http://www.arduino.cc/en/Hacking/LibraryTutorial

I think the community should be the 'check' so that we could have a flag for 'does not work' for instance. And maybe also a vote up/down.

We could add categories to the libraries. Math, Data Structures, Timing, Processing, Hardware Abstraction etc etc etc.

Sooo...

I've purchased a domain/webhotel and started on the UI: http://www.arduinolibraries.net/
Please come with ideas and feedback!

Nothing is carved in stone, and I'm not in favor of any one solution.

I for one think this effort can be of great service for the Arduino community.

I've created a first draft of an Arduino Library Markup Language
Sample XML: http://alexanderbrevig.com/hosting/arduinolibraries.net/ArduinolibML.xml
Sample XSD: http://alexanderbrevig.com/hosting/arduinolibraries.net/ArduinolibML.xsd

Thoughts and feedback VERY much welcome.

@beo6 since you know C++ you'll need just a minimal amount of time to understand Java
These types of applications is what works best on Java, IMHO.

I suggest using "tags" instead of "categories".

They're intended to behave like tags, but I've got to add a tags input box to the form for entring new libraries.
True That.

Nice work.

i didn't expected so much response for my application especially from people that have more than 1000 posts.

But isn't there a upload field missing?

Will look to change my application as soon as the XML format is right.

So i will change it that every uploaded zip needs only 1 folder with the libraryname and inside that the library cpp files with examples.

I am no expert in C++ and don't want to learn Java right now.

I for one would ask why the Arduino IDE is Not! in C++.

But this might end in a programming language war.
Everyone should code in the language that he likes most. :wink:

@AlphaBeta´s Edit:
I don't really understand what idea you mean that is stupid. :-?

But isn't there a upload field missing?

Will look to change my application as soon as the XML format is right.

There's probably several things missing. But, because I would want to keep track of versions, the 'upload a library' (which will actually be 'provide an URL to your zip archive) will be an action that has to do with a library version. So I think it's best to have a separate form for adding a new version of a library.

Feel free to have a look at the XML and rewrite it as you would like it to be. The most important features for me is:

  • any number of authors
  • a short library descriptions
  • any number of library versions, with a version number, a .zip and a short description

So i will change it that every uploaded zip needs only 1 folder with the libraryname and inside that the library cpp files with examples.

That is brilliant.

@AlphaBeta´s Edit:
I don't really understand what idea you mean that is stupid. :-?

If you thought I was an ass for suggesting a rewrite, I'd like to apologize. That's all.
Keeping this project a separate endeavor is probably not a problem at all. It is the service which is in focus, and when we get used to being able to easily download libraries, and have an XML and a backing website in place, adding a downloader to the IDE won't be hard at all.

I for one would ask why the Arduino IDE is Not! in C++.

It's because the Arduino team used an already developed multiplatform/open sourced IDE used in the Processing project. This allowed the Arduino project to come to light much quicker then if they had to create the IDE from scratch. Nothing to do with language wars. :wink:

Lefty

Great tool!!

I use something similar into a GIS open source program: QGIS. It has a similar tool to download scripts and tools from the official repository, but also from three-part repositories... It also show an advise when new version from a library is available. Other interesting option: it allows you to install libraries and also remove them. Just only some ideas that could be interesting for this library download tool for Arduino IDE.

Thanks for this tool!

I am really interested if my tool even works for anyone.

I just tested it on a different PC and the application didn't start at all. :-/

Now switching the development tools and will see if it works better.
Nice side effect is a Development environment where i am more comfortable with.
Just solving the last problems.

Of course i am not thinking someone is an ass just for suggesting something. Even a rewrite.

Maybe when i am more comfortable with building GUIs with wxWidget i might be able to make a small C++ IDE for Arduino. :-?

The remove libraries feature is already planned.

I know the argument for JAVA for the IDE comes with the understanding that it is what is required for cross platform development... but C++ with WxWidgets seems to be well supported on multiple platforms... so... good choice. :slight_smile:

It seems that a WxWidgets solution should be able to work for Linux and Apple in additional to Windows. Good Luck.

so i got my development environment working yesterday.

i compiled it and it also works now on the PC where the application didn't started before.
So i think this is a good sign.

i also got some code running to extract the library zips directly with the zip library. Just need code the folder structure creation from the zip. (currently only in the SVN)

when that code works some library zips might no longer be supported because of different folder structures but i think the advantages outweigh the disadvantages in the long term and it will be easier to compile on different platforms without the need of 7zip.

After that i will look to parse the XML from AlphaBeta. It would be nice if i could get a working example from this XML with at least 1 working testentry.

If someone wants to help in programming this, i can create him a SVN login with write access of course. :slight_smile:

I hope i do not Spam here or we should talk outside of this forum.

I'm interested in contributing to this, maybe we should set up a group somewhere, so we can talk and decide how to proceed.

I'd like to focus on the web/backend and maybe you should focus on the client?

We could continue our discussion here, so all can contribute with feedback/questions etc etc.

What you think?