Encountered lousy boo-boo in arduino IDE generated compilation commands

bperrybap:
The way the IDE does its build now, you pay the price for separate module compilation,
without gaining the benefits like the ability to avoid re-compiling modules that have already been compiled.

That really depends on your definition of the word "now" :slight_smile:

Yes, that's how Arduino 1.0 works.

But the code on github, soon to become Arduino 1.0.1, has a patch I wrote many months ago which avoids recompiling code. It works more or less like you described, except it doesn't retain the .o files or a .a library for each board setting. It simply uses the .o files left over in the temporary directory.

This feature has existed in Teensyduino for nearly 1 year. Very soon it will be in Arduino.

Placing all the .o files for each library into a .a archive probably doesn't provide much benefit. If the archives were retained in a non-temporary directory, it could speed up the first compile when you begin running Arduino, or the first compile after changing to a different board (that's been used previously). That might be nice, even though it's a much less common case, but accomplishing that would take a pretty substantial patch. I doubt anyone will actually work on such a patch, and even if they did, I believe David probably would be reluctant to accept it.

One other thing to consider is where Arduino can write files. Long ago, it would create an "applet" subdirectory within the sketch folder and put the compiled files there. But that behavior caused all sorts of problems when compiling the examples, because many schools would install Arduino on a read-only networked filesystem. To solve that, the temporary directory was used for examples, but applet was used for normal sketches. Eventually, the "applet" subdirectory was removed, so Arduino would only attempt to write to the temporary folder. Writing all generated files to the temporary directory greatly improved satisfaction with Arduino among security conscious users, and the linux distros who now package Arduino to work across several "standard" locations.

Any proposal to store compiled archives must carefully consider the restrictive shared network filesystems which are used by many academic institutions. A patch which returns Arduino to the "bad old days" of writing within its own directories is certain to be rejected.

I was saying the copying of the source files to this build area should not necessary.
It should be possible to compile the code with the source files in place
and redirect the output objects to the very same place they are currently are.
(avoid the copy of the source files)

The library and core source files are never copied to the tempoary build directory. Only the sketch source is copied, and for good reason...

Arduino concatenates all the .ino & .pde files (if there's more than one) and does some preprocessing, mainly to automatically add #include <Aduino.h> and function declarations, which allows functions to call each other regardless of their order within the files.

Perhaps that preprocessing step could be done entirely within memory, with the result fed into the compiler via a pipe? But that would require quite a bit of extra java code. It's currently written in a fairly simple way, where all the .ino / .pde files are turned into a single (temporary) .cpp file, before the "Compile" step is even begun. While compiling, gcc is run using pretty much the same code with works on a paradigm of files being compiled to produce other files. If the preprocessed output were never written to disk, not only would the preprocessing code need to be changed to more complex code to manage the whole thing in memory, but the compile code would need to run gcc differently for one part where the input isn't a file, and the entire structure within Arduino would need change to pass the in-memory copy between the "Sketch" and "Editor" sections and the "Compile" object.

That would be a monstrous patch, only to avoid writing one temporary .cpp file!

More importantly, as the Dr pointed out in the OP,
the way the IDE works, you are stuck having to add includes to the main sketch
in order to get the include path setup to allow other modules to compile.

And that is the main thrust of the the Dr's comment in the OP.
It was that in order for a module to use any
code form a library, the main sketch must include a header file from that library
to get the IDE to do some "magic" things and set up the include path.

I have often considered writing a patch to fix this. It doesn't look like anyone else is ever going to do it....

I'd probably just reuse the existing parser that preprocesses the .pde / .ino files to extract a list of the #includes from all .cpp files, as if they'd been in the user's sketch. But if there's something special it should do, please enlighten me?