Library usage is nonstandard

Projects Programming
1

I do not understand how the IDE uses libraries.

Consider: I have a folder "flounder" in which I put all my .h files that I have written. Fine.

I then to #include <whatever.h>.

It appears that this #include fails, and furthermore there is no error message indicating that it has failed.

So I go to the sketch menu, and import the library. I get a randomly-placed #include in my source, but nothing seems to happen.

What is extremely peculiar is that it appears to make a COPY of the .h file in the current project directory. In what world could this POSSIBLY make sense? It appears that if I make changes in the library, the changes will not appear in the compilation of my code, because it seems to be reading the "local copy" that has been made.

Furthermore, I had a definition
#define Whatever(a, b, c) ...definition

and I changed it to an overloaded function:

Thing Whatever(int8_t _a, int8_t _b, int8_t _c) { Thing t; t.a = _a; t.b = _b; t.c = _c; return t; }
Thing Whatever(const Whatever & value) { Thing t; t.a = value.a; t.b = value.b; t.c = value.c; return t; }

Please note that this is a simplified example, a Thing is not really a class, so don't tell me that I'm using C++ incorrectly and these should be Thing constructors. There is a deep reason that the overloaded function approach makes sense here.

When I invoke it as Whatever(t) I get the error message "Macro Whatever takes 3 arguments, and you have given only 1" (or something along those lines). But there is no longer a macro! In an attempt to repair this, I delete the copy of the .h file that was made in my project directory, re-import the library from the Sketch menu, and try again. Same error message. This is impossible. There is no header file ANYWHERE that defines the macro "Whatever". It is now an overloaded function, which ideally is inlined (but that is not critical). How can this be? How can the compiler use a nonexistent, dead, obsolete, irrelevant definition when I have changed the header file?

All I want is that this compiler works like every other C/C++ compiler in the Known Universe, and that the "import" just adds the directory to the header search path! Otherwise, it becomes impossible to manage libraries, if I don't know how many projects have their own private copy of an obsolete version, or even have some magical cache of some obsolete version (perhaps in the .ino file? If so, WHY????). If it has a cache, it is not doing a competent job of cache management, since in a sane world it would compare the timestamp of the file in the library location to the timestamp of its cached copy, and flush any obsolete definitions from its cache.

At the moment, I am totally blocked from progress because the IDE does not do reasonable things. I don't even understand what it IS doing, nor have I been able to find any documentation of what it does. Note that as one experiment, I changed a line in the .h file from "#define" to "#refine", but I get no error messages from introducing this syntax error. Huh?

Note that I have been programming in C++ for 20++ years. I am not a newbie at this. Someone suggested that you do weird things so newbies don't have to worry about something-or-other. That may be fine for newbies, but I need a "professional programmer" option that tells the IDE to NOT do "newbie-friendly" things, and just work the way it is expected to work by any professional programmer's expectations.
joe

2

Note that as one experiment, I changed a line in the .h file from "#define" to "#refine", but I get no error messages from introducing this syntax error. Huh?

What does the quote above tell you? Does your 20+ years of experience tell you that the compiler is not reading the file you think it is reading. If not, why not? When you save the new header file, is it being saved where you think it's being saved (e.g., does Save As show the expected directory?) If you do File --> Preferences, is the sketch directory where you thought it would be? Have you tried going to the directory where the IDE is invoked from and looked in the Libraries directory to see if your library is there? Does anything different happen when you surround the #include directive file name with double quotes verses angle brackets? Could it be that this compiler doesn't work like every other compiler in the known Universe and, as such, may require reading the docs of how it treats its compile files?

3

What is extremely peculiar is that it appears to make a COPY of the .h file in the current project directory. In what world could this POSSIBLY make sense?

This world: The IDE will compile your "sketch" (code) without you saving it (ever). It looks like an "in memory" compilation, but of course that doesn't actually happen. Also, you can compile a sketch without saving changes and get the changed code.

For both of these to work the IDE has to write a copy of your code to a temporary directory and compile there.

In the process it pre-processes your main sketch a bit, adding function prototypes to save you having to learn about them .

If you believe changes to libraries are not being processed can you please give a step-by-step sequence to reproduce?

I personally develop libraries by having the library code open in a text editor (not the IDE) and just saving changes as I go and then run a test sketch in the IDE. I have never had problems doing that.

I think there might have been problems earlier with it not detecting changes.

Please note that this is a simplified example, a Thing is not really a class, so don't tell me that I'm using C++ incorrectly and these should be Thing constructors.

I'm sorry but this is coming across as a rant. You need to post concrete examples. Not vague "my code looks something like this" stuff.

How to use this forum

If you describe exactly what you are doing I (and others) will try to reproduce your problem and explain what is happening.

4

flounder:
If it has a cache, it is not doing a competent job of cache management, since in a sane world it would compare the timestamp of the file in the library location to the timestamp of its cached copy, and flush any obsolete definitions from its cache.

Just to test your claims, I looked at one of my libraries (Regexp) and loaded an example sketch from it, and compiled it (thus loading up this cached copy).

Then I used gedit to change the Regexp.cpp file to add:

#error foo

... to the start of it. Hit compile again and got:

/home/nick/sketchbook/libraries/Regexp/Regexp.cpp:1:2: error: #error foo

So clearly that is working.

Just BTW, where are you storing your libraries?

... I don't know how many projects have their own private copy of an obsolete version ...

Well, personally I name my files in a logical way. For example my Regexp library has Regexp.cpp and Regexp.h. I suppose I'm hoping another library won't also have those names, but if they did you would have the same problem in a normal C++ development environment.

5

flounder:
Note that I have been programming in C++ for 20++ years.

You are going to have to unlearn a few things.

Splitting Sketches. You can move code out of the .ino file into .h and .cpp files located in your sketch folder and include them in the sketch using the "..." syntax.

#include "myfile.h"

As you might expect, the compiler includes the .cpp file with the same name as the included .h but the presence of the .cpp file is optional. As you might not expect, the files must reside inside the sketch folder. You can not easily share code amongst your sketches using "..."

Sharing code. A library is what you get when you create a folder called 'libraries' inside your sketch folder and create a subfolder to locate your own .h and .cpp files.
E.g. To create a myClass library

/MySketchbook/libraries/myClass
myClass.h
myClass.cpp

You can then include the library in any of your sketches using the <...> syntax.
mySketch.ino
#include <myClass.h>

Your own library can refer to other libraries, using <...> and to other files located inside it's own library folder using "..."

If you have done things right, your library should show up under the Sketch/Import Library menu, within the Contributed section. You might have to restart the IDE to register the new library...Or that might just be a ritual of mine, as my sketch folder is on a network drive which frequently gets disconnected.

Probably worth mentioning what the Sketch/Import library/Add Library menu command does. Many libraries are distributed from git hub in the form of zip files. The Add Library command unzips the subfolder tree into the libraries folder, making the library available to your sketches, or at least it attempts to. Whenever I have tried to use the feature, the moons have been somehow mis-aligned and it has failed. Now I know what it is trying to do I just do it by hand, which I think is easier, quicker and less confusing.

Creating and modifying library files is a bit bizarre, due to the copying that goes on when you try to open a file in a library folder.
One way is to adopt a consistent release workflow.

  1. Create a sketch folder and a .ino file to act as a test harness.
  2. Create .h and .cpp file (tabs) in the same folder and include in the test sketch using the "..." syntax.
  3. When you are done developing your library files, release them into your libraries folder, by creating a subfolder and copying the .h and .cpp files into it. Restart the IDE and your new library should show up under the import library menu.
  4. When you want to modify the library, open the test harness sketch and modify the files there and re-release to your library folder when you are done.

That is quite some faff to change a few lines though.

Another way is to use a syntax aware text editor, like Notepad++ or TextWrangler to work on your library files, while you edit and compile your .ino sketch in the IDE.

Another way, the way I prefer, is to use a better IDE. My favourite is Atmel Studio 6.1 with the rather excellent Visual Micro plug in for Arduino. It can be a totally free solution, although I chose to pay for the VM debugger which I hardly ever need to use, just to support Tim who runs the project. There are also working plug-ins for Eclipse and for XCode. These alternatives depend on the Arduino toolchain and retain some of the oddness, but usually offer a work around. For instance, I can open library files from the library folder as if they were any other .h, .c or .cpp file, alongside the .ino and supporting files of my project.

Hope that helps ease the frustration of your adjustment period.

Someone suggested that you do weird things so newbies don't have to worry about something-or-other.

I don't believe you need to condescend people to educate them :wink: