I am using ESP32 by espressif systems V2.0.17 for a project that requires IR_remote library that can only use up to this version of ESP32 (latest being 3.0.7 that didn't compile).
I don't see -j as an option for xtensa-esp32-elf-g++ to distribute jobs among the cores or threads on a CPU but I remember when I was using ESP-IDF V3.3 I had to add a -j option to use all cores/threads, which sped up things by a lot.
As I am working on ESP32 Arduino today, I observed that only 2 out of 20 threads were on high percentage when I was compiling ESP32 code in Arduino IDE 2.3.3, very slowly.
I've done a very large project with ESP-IDF several years ago that required lots of code I wrote besides all includes. I bought my machine so it could compile several times faster than the previous machine I was using. I'm hoping to utilize all threads under Arduino IDE.
I don't see a -j option for the g++ in either 2.0.17 or 3.0.7. Maybe you're thinking of the one for make?
The arduino-cli back-end has a similar flag: --jobs. It should default to zero, which means "number of cores" as seen by the Go process
// Spawn jobs runners
var wg sync.WaitGroup
if b.jobs == 0 {
b.jobs = runtime.NumCPU()
}
You'll get the benefit of parallel compiles mostly with libraries and the core. Maybe if you have multiple .cpp in your sketch? Not really with .ino, since those all get glued together into a single file.
You could try using --jobs 1 to force no parallelization; see how much worse that makes it.
You can see that all threads were used during a burst of about 30 seconds. But if you focus on the top row, middle and rightmost threads, those were the ones that were running to compile the IR_remote library and my code alone (guessing the threads belong to one core), before the compilation of the core and esp32 libraries started. Unfortunately the first part, the supposedly easy part with one contributed library and my code (a few dozen files) took 60 seconds while the second part with core and esp32 libraries only took 30 seconds. I did a clean build after quitting arduino, deleting the build folder in temp folder and restarting arduino and recompiling.
So it seems that the IR_remote library plus my code (short) took 60 seconds. Is there any way to optimize this portion of the compilation?
I'm going to ask you to provide the full verbose output from a subsequent compilation.
This procedure is not intended to solve the problem. The purpose is to gather more information.
Please do this:
Select File > Preferences... (or Arduino IDE > Settings... for macOS users) from the Arduino IDE menus.
The "Preferences" dialog will open.
Check the box next to "Show verbose output during: ☐ compile" in the "Preferences" dialog.
Click the "OK" button.
The "Preferences" dialog will close.
Select Sketch > Verify/Compile from the Arduino IDE menus.
Wait for the compilation to finish.
Select Sketch > Verify/Compile from the Arduino IDE menus. ⓘ I am intentionally instructing you to compile twice. The reason is that I want to check whether Arduino IDE's compilation caching is working on your system. The first compilation is to ensure that the second compilation will be performed with a populated cache in place.
Wait for the compilation to finish.
Right click on the black "Output" panel at the bottom of the Arduino IDE window.
A context menu will open.
Select Copy All from the menu.
Open a forum reply here by clicking the "Reply" button.
Click the <CODE/> icon on the post composer toolbar.
This will add the forum's code block markup (```) to your reply to make sure the error messages are correctly formatted.
Press the Ctrl+V keyboard shortcut (Command+V for macOS users).
This will paste the compilation output into the code block.
Move the cursor outside of the code block markup before you add any additional text to your reply.
Click the "Reply" button to post the output.
In case the output is longer than the forum software will allow to be added to a post, you can instead save it to a .txt file and then attach that file to a reply here.
Select the .txt file you saved from the "Open" dialog.
Click the "Open" button.
The dialog will close.
Click the "Reply" button to publish the post.
Alternatively, instead of using the "Upload" icon on the post composer toolbar as described in steps (5) - (7) above, you can simply drag and drop the .txt file onto the post composer field to attach it.
Great! From looking at second compilation.txt, I concur that the caching features were working as intended.
However, I am wondering about the fact that the "IRremoteESP8266" library was not already cached cached at the time of the "first compilation". The libraries are cached on a per-sketch basis, and that cache is reused whenever you compile the sketch (unless the cache is invalidated due to modifying the library source file, or changing the board configuration from the Arduino IDE menus). The cache is stored in your system temporary folder, but Windows doesn't ordinarily automatically clear the temporary folder and Arduino IDE only clears it if 30 days pass without you compiling the sketch. So you should only need to compile a given library for a given sketch for a given board once.
Do you have any idea about why the library was not cached at the time of the first compilation? Did you compile this C:\Users\tarza\OneDrive\OneDrive Documents\Arduino sketchbooks\Contract_work\Pinnacle Systems LLC\CommonAcControl sketch for the "ESP32 Dev Module" board prior to the "first compilation"?
15 s seems like a reasonable compilation duration to me. A 90 s duration is annoying, but if you only need to suffer that for the first compilation then it isn't as significant a problem.
I deleted the temp folder before starting IDE so first compilation was done with no prior compiled libraries.
CommonAcControl was my simple sketch that contains less than 100 lines of code. It uses IR_Remote library.
What's bothering me is that, after making a superficial change to the char array in one serial print, I am back at compiling everything again for 90 seconds, no such luck as using cached files from second compilation, even though only file changed is my sketch.
When I open my sketch by itself, and compile, update sketch, recompile, expected behavior is observed, ALL compiled library files just default to cached.
When I open my sketch, and then right click on an IR_Remote library header file and choose "Go to definition", this header file is opened in a tab in the IDE. Here is where every single library file and core gets recompiled every single time I make a trivial change to my sketch.
This behavior goes away after I close the header file tab in the IDE. The first compilation after I close all header file tabs starts using cached files.
My guess is that when I save my file, even those supposedly read-only tabs that display header files get saved so they are "ruined" and the whole library gets recompiled due to the update to the last modified time stamp of these read-only header file tabs!
I think this is a bug. Read-only means read-ONLY. I installed IR_Remote library on OneDrive and was able to monitor such time stamp update from OneDrive notification every time I update my sketch superficially and compile. At on time have I modified any library headers.
What's bothering me more is why the esp32 core and library also are recompiled when a contributed library is "modified".
Now I know what's causing me 90 seconds of time, I'll just say that Arduino IDE 2.x sucks and its useful features like definition lookup are not worth using
Here is a screenshot of the IR_Remote library in reverse order of modified dates. You can see that all those 11/26/24 modified dates were the ones that were open in Arduino IDE tabs (as read-only) and the ones below the red line were when I last installed the library (unchanged) because I didn't open those in Arduino IDE tabs. I think this is definitely a bug with dire consequences. How do I report it?
This procedure is not intended to solve the problem. The purpose is to gather more information.
) on the post composer toolbar:
