The typical scenario for a bigger project is like this: click compile or upload. Wait for Arduino to scan all the headers. Wait good 2 minutes for it to compile libraries. Then it says "Compiling Sketch" and I see "Missing semicolon on line 30".
Whether it will reuse the compiled libraries or recompile them from scratch seems to depend on wind direction and phase of the moon. Regardless, it takes quite a while until I get to the sketch compilation phase, where my errors are shown.
Would it be very hard to swap it so that the sketch files are compiled into objects first, and only if successful, the process proceeds to bulky libraries?
Probably not. I try to start with a small sketch, add empty stubs for known functions, then start filling them in.
It may be less obvious how to do that for someone who does not have decades of experience, but the old phrase of divide and conquer or start small come to mind.
It might be worth considering VSCode + PlatformIO. I started with the Arduino IDE (1.8.13) and I haven't tried 2.x. Now I only use the Arduino IDE for small projects. I find PlatformIO more powerful and the builds are much faster apart from the first time. It seems pretty good at not rebuilding libs / anything that hasn't changed since the previous build.
I just made a change to one source file in a project with 50+ source files (.cpp + .h) and I'm using a quite slow 10 year old macbook running win 10. It took 32 seconds to do the build.
Arduino IDE 1.x clears the compilation cache when you close the IDE. So you will have to wait for a full compilation of the sketch each time on the first compile after you start the IDE.
Arduino IDE 2.x does not clear the compilation cache when you close the IDE. So you should only have to do a full compilation of a given sketch for a given board once, until such time as external changes (e.g., system temporary directory is cleaned, update the platform of the board via Boards Manager, update a library dependency via Library Manager) might force the cache to be cleared.
Are there any non-ASCII characters in your username on your operating system, or in the path of your sketchbook folder (as defined by Arduino IDE's "Sketchbook location" preference)? I ask because there is a known bug in Arduino IDE that causes the caching to not work when the path under which a library is installed, or the path under which the compilation cache is stored contains non-ASCII characters.
This is easy for a simple project. If I'm debugging a handler for EspAsyncFsWebServer, I need said server,WiFi, Networking, AsyncTCP, LittleFS, ArduinoJSON, and a couple more to even get it to compile. No simple way to add them later.
Possibly, I worked with a project for it, but I fell in love with Arduino's ease to create a project, how it automagically handles library dependencies, how there's about zero digging in stuff "behind the scenes". I'm finding setting up a PlatformIO project with multiple dependencies from scratch quite daunting.
Arduino IDE 2.3.2, arduino-esp32 3.0.4 from Espressif, no non-ASCII characters in my username (preinstalled Windows so username = oem), the path is the default C:\Users\oem\Documents\Arduino
There is a learning curve, but it didn't take me long to become familiar with it. However, from other's posts, Arduino IDE 2.x shouldn't exhibit the slow builds of 1.x, so it sounds like switching to PlatformIO to improve build speed isn't necessary
Yeah, didn't do any measurements or statistics but it felt to me like changes in a file that #includes a library result in recompiling that library and all its dependencies. Albeit I won't swear on it; I think I saw at one point roughly 3/4ths of EspAsyncFsWebServer recompiled and last few files pulled from cache, so I really have no clue how that works.
BTW, is there a valid reason why the sketch is compiled last? Or is it just "it's written that way"?
In order to compile the sketch, you need to find all the included headers. When you ask for WiFi.h, which one is that exactly? That's what the "detecting libraries" stage is about, and what is slow, even when everything is cached. In that case, the .ino is always recompiled, but the libraries and core take "no time". Linking is also a chunk of time, even if "nothing changed".
FYI @sharpfang, the Arduino developers have been doing some work to reduce the time it takes to perform the "detecting libraries" stage mentioned by @kenb4:
Don't forget there is a lot happening on a modern computer. There are internet connections to check for github updates and files that were cached for some reason become uncached. I currently have 474 processes running with over 4,000 threads so I think it's a small miracle that anything gets done quickly.
I didn't know how to time mine, but it seems that indeed 'detecting libraries' takes lion share of the time.
After compiling and having everything cached, I only changed whitespace in my sketch's files, and compiled it again. Compiling a file takes roughly constant time (really not sure why, they differ in complexity quite a bit) - and so I counted calls to g++ and the 'Using cached..." messages. Here are my results:
