How to Efficiently Develop Libraries for Arduino (Advance Users POV)

Projects Libraries
1

I hope you are all doing well.

I am developing many libraries for Arduino and I am tired of surprises here and there specially when it comes to folder and sub folder related issues. I kind of hope a guide is written for advance users so that they do not waste time digging information in various places.

  1. IDE is not the best choice, so I started using Sublime and Stino to solve the issue. It may not be the best but considering my limited experience so far, it has been way better than Arduino IDE. I will be happy if anyone has a better suggestions. Sublime and Stino run pretty decent only issue is that rarely files do not get compiled after you have changed them. Some compiler messages are also not shown but I guess it is no big deal. I have seen the Arduino list for different IDE but there are so many mixed opinions about them. I tried to setup few of them but gave up int he process (Not recently though)

From here let's assume you want to develop a library and you want to integrate SVN/Git with your progress. First question will be right after you want to start.

  1. How do I do it?
    S1: I make library folder as a place to keep my working copy and then edit the library folder directly. the issue with this solutions is that by doing so if your library has multiple .h files and folder, you will not be able to access them from an .ino file. For a finished library this seems fine because you know exactly what user is going to use and the functions are all tested, but if you are developing you need to test the internal functions and also the interface may change so by early estimation of functions used by user, you will rewrite many lines of boring code (wrapper).

S2: Sketch file become working copy and you develop library from there.
The issue with solution is that now you cannot have any folder structure and you will need to change all the include lines after you want to make it a library and save it in Library folder. When you save it in library folder, you need to either restructure your repository folder or you need to have a different folder for codes that already become library. First approach is very time consuming, and in second approach you loose the history of the file (there may be a work around this but I cannot think of easy solution, I am not a pro SVN, Git user).

S3: I have no idea.

  1. The unusual rules of sub folder support has changed over the years and it makes it very difficult to know what is the current level of support. So far from my understanding and testing:
    a) Sub folder can exist in library folder, but those files are not accessible from .ino file. (I tested this and it did not work, but I have read that it is possible)
    b) Sub folder in sketch cannot be included.
    c) Arduino Library folder name and the main .h file should match so that IDE can include it.

I think Arduino has a lot more support for new users than for advance users and it is kind of sad. I know a lot of advance users have found solutions for all the problems mentioned above but I have had a very hard time finding a legit one piece article that cover all the stuff.

Sorry if the message is a bit negatively written, I just figured out about 3)a) and I changed one of the very huge library thinking that it would work fine.

2

I commented in an older topic how I develop libraries

3

alirezasafdari:

  1. IDE is not the best choice, so I started using Sublime and Stino to solve the issue. It may not be the best but considering my limited experience so far, it has been way better than Arduino IDE. I will be happy if anyone has a better suggestions.

You haven't told us what this "issue" is so it would be impossible to suggest a solution.

alirezasafdari:
S1: I make library folder as a place to keep my working copy and then edit the library folder directly. the issue with this solutions is that by doing so if your library has multiple .h files and folder, you will not be able to access them from an .ino file.

Why would you not be able to access them from an .ino file?

alirezasafdari:
S2: Sketch file become working copy and you develop library from there.
The issue with solution is that now you cannot have any folder structure

If you're using any relatively recent version of the Arduino IDE then you can indeed put source files in a subfolder of the sketch folder. They only need to be all within the src subfolder of the sketch folder. You can have any folder structure you like under src.

alirezasafdari:
3) The unusual rules of sub folder support has changed over the years and it makes it very difficult to know what is the current level of support.

Luckily it's all clearly documented here:

alirezasafdari:
a) Sub folder can exist in library folder, but those files are not accessible from .ino file. (I tested this and it did not work, but I have read that it is possible)

That's wrong. You can access them but there is a bit of a trick. You need to first include one of the files from the root source folder so that the library folder will be added to the include path. Let's say I have this folder structure:

Foo
|_src
|_Foo.h
|_utility
|_Bar.h

I can include Bar.h from the sketch file like this:

#include <Foo.h>
#include <utility/Bar.h>

However, the best library design would use the root library source folder to contain the user facing files and subfolders to contain source files used internally by the library.

alirezasafdari:
c) Arduino Library folder name and the main .h file should match so that IDE can include it.

This is not a requirement but it is best practices because if there are multiple files found that match an include the one which matches the library folder name will have include priority. If you avoid using generic names for your files then that situation would be very unlikely to occur.

4

Thank you pert. I just had a glance and I think it should be all clear now.

For the part about IDE, I thought it is a common thing. drop down suggestions, syntax coloring and the small detail that becomes a big deal when the code is getting long.

Anyways thank you so much for the answer. I will read through it more carefully ASAP but there are so many things that I loved about your answer.

5

I wish I could see the link you sent in my google search because it answered all the questions. The trick was using the src folder and utility.

All my observations were made not using src and utility, that is why the whole thing is wrong :smiley:

BTW I think

#include <utility/Bar.h>

should be actually

#include <Foo/utility/Bar.h>

At least based on what I understood from the link.

Thank you so much pert. You made my day.

I am going to try what I understood and I am so happy right now.

6

alirezasafdari:
I wish I could see the link you sent in my google search because it answered all the questions.

Yeah, I think the GitHub wiki files don't have a very high Google page rank so they are harder to find. There is some really good information in some of the other pages of that wiki also, though maybe not of interest to you. That's where the more advanced documentation that is meant for developers rather than average Arduino users is published.

alirezasafdari:
BTW I think

#include <utility/Bar.h>

should be actually

#include <Foo/utility/Bar.h>

At least based on what I understood from the link.

No, because when you do #include <Foo.h> it adds Foo/src to the include path so you only need to specify the path relative to that folder. A great thing about computer science is that it's so easy to run experiments to test our hypotheses.

7

I have changed IDE and I thought it is a good idea if I share my experience here.

Apparently the issue was not how Arduino IDE works after I read the links from pert. The issue was Stino plugin in Sublime text. For some reasons it works differently and I spent some time to figure out the differences but I eventually give up. But I can confirm as of today the Stino and Sublime do not work same as arduino IDE. I had a sketch compiling fine with src folder while sublime stino could not compile the file and it would just look for source files. The worst part was that I had to close it and open it again otherwise it would look for source files forever and you cannot build any project afterward.

I then proceed on choosing a new IDE and my love for visual studio kind of pushed me towards visual micro extension in both Atmel Studio and Visual Studio.

I installed both extensions and did a comparison between the two. The files are included in both IDEs very easily and they do have a feature named "Deep Search Include" which make the including part same as Arduino. It can even include files which are not placed in your src.

There are differences though between Atmel Studio and Visual Studio which I think are important.

Visual Studio:
i) Using original theme is kind of sad. Your whole code is almost written in same color. Worst part is that things like "Serial" are in the same color as numbers and user defined objects and so on. (There are solutions to this and I have not found the best one yet)

ii) Visual Studio is lovely because it does not add an actual copy of the files to the project. Instead it will just go and use the file wherever it is located. This feature is really handy if you have version control. You can also create a copy in your project if you really need a duplicate. So you have the best of both worlds however the second one is more time consuming than the first one.

iii) It runs and compiles faster than Atmel Studio. I can understand why it runs faster but cannot understand why it compiles faster.

Atmel Studio:
i) Awesome color code. It is lovely, your code has different colors and in my opinion it is very helpful.

ii) It always create a copy of files. This means that you should always make sure every copy is the latest copy across all places which is using the file. I could not find a work around for this but I did not dig hard enough. (although the library path does not have this issue based on what I read online)

iii) It runs and compile slower than visual studio.

Overall on Micro Visual:
i) the debugging solution is fantastic. I have not used it but I have been doing it manually for many times in arduino projects. So if it works as well as the documentation says which so far has, it is fantastic.

ii) It is very mature and I think they have done a fantastic job. It is a bit hard to dive in specially if you are first timer in Visual Studio, but on the positive side remember all the projects you can do with Visual Studio. So it is kind of worth it if you are going to expand the type of code you write. I had not used VS since 2010 and it was a little bit hard for me to get the short cuts sorted out.

iii) The Intellisense is very interesting and kind of smart. Options are too many but still better than nothing.

I am just worried on the financial sustainability of Micro Visual but i deeply hope they can survive because they have done a nice job.

8

I could have been wrong about Sublime and Stino. For some reasons a short test today seemed to work all well. I am already balls deep in Visual Micro and I can see its benefits already, so I am not going to spend time to see what happened that day. I have also deleted the testing folder from my computer. Just want to make sure other readers are not misled by my posts.