I use a build_opt.h file in the sketch to set options in the processor core and or/library.
The problem is that swapping between sketches with different build options fails to work correctly because of caching.
The clumsy way around this is to swap to a different processor and compile to force a cache flush. Closing the IDE and opening again no longer clears the cache.
I understand that the Arduino IDE is for "beginners" but with new features like debugging etc it seems to be moving away from the KISS approach. It therefore seems an appropriate time to either delete the build_opt.h capability (yuck!) or simply add an "advanced" option in preferences to clear the cache before each compile.
I should make clear that this is not anything Arduino created (in fact Arduino has historically rejected proposals to add an equivalent capability to the IDE).
This was implemented in the 3rd party STM32Duino boards platform:
It might be there are some other 3rd party platforms that did the same.
I have an alternative proposal: instead of an option that sets this behavior persistently, add an advanced command to Arduino IDE that clears the cache before compiling. You could assign a keyboard shortcut to this command for the quickest access, or else just select it from the command palette.
What do you think about that approach?
Although I don't have any experience with this build_opt.h file, I also sometimes miss an easy cache clearing mechanism. I have never found out what causes it, and never experienced it myself, but people periodically report mysterious "undefined reference to ..." errors which are caused by the cache being corrupted in some way. This happened with Arduino IDE 1.x for years as well, so it is nothing new, but the problem could be resolved by restarting Arduino IDE 1.x. Although the persistence of the cache in Arduino IDE 2.x is actually very beneficial overall, in this specific situation it is a bit annoying.
For my use case, being able to trigger the cache clearing on demand would be much better than having to change the settings to clear the cache, do a compilation, then change the settings back.
Even for those who use the build_opt.h file, it seems to me they would still like to use the cache when possible for the sake of quicker compilations, clearing it only when needed.
I understand the explanations of ptillisch, thank you.
it would be simple to empty the cache when testing the presence of the build_opt.h file, wouldn't it? it's automatic, and it solves the problems of the platforms that use it.
even if an option in the preferences to activate this feature is welcome
The STM32Duino boards platform uses the script I linked to in my previous reply to create an empty build_opt.h file if one doesn't exist in the sketch. So a presence check won't help.
However, I think you are on the right track generally. The solution of making the caching system smart enough to always automatically clear the cache when relevant files have changed is the ideal one. This was also proposed here:
I think the system already does a pretty good job of this, but evidently not so in this case. The fact is that this is quite a hack. The build_opt.h file is not actually a header file. It is certain this use case was never considered during the development of the compilation caching system.
as you write, STM32 use this config file.
if the cache is not erased in this use, it is a source of error during compilation and it is complicated to empty the cache, or not yet possible with ide 2.0 which should be quickly imposed.
the fact that it is named .h without being a real header file, is due to the limitation if I understood the history correctly that the ide (1.x) only accepts .h, .c, .cpp, .ino.
so it's a hack yes, but vital for STM32 and useful for ESP32 since they can use it too.
although this approach was not originally thought of, it has become an advanced option over time, so it should be added for ease of use.
personally I use it to switch to complex libraries some options without modifying the library files (example TFT_eSPI of bodmer between different projects and different screens) and with the ide 1.8.x it works very well, even if you have to restart the idea in case of change of project).
deleting the cache when the existence of build_opt.h should work without much addition and without impact.
This is incorrect. bodmer mentioned one option already:
I'll provide detailed instructions:
Change a custom board option under the Tools menu to an arbitrary selection.
Select Sketch > Verify/Compile from the Arduino IDE menus.
Wait for the compilation to finish.
Change the custom board option back to its previous selection.
The cache is cleared when you compile after changing the option. So this provides one option for clearing the cache via the Arduino IDE UI.
Another option is to manually delete the cache:
Select File > Preferences from the Arduino IDE menus.
Check the box next to "Show verbose output during: ☐ compilation".
Click the OK button.
Select Sketch > Verify/Compile from the Arduino IDE menus.
Wait for the compilation to finish.
Examine the contents of the black "Output" panel at the bottom of the Arduino IDE window. You will see some lines that show the path of the caches. These are in two locations:
The cached libraries are in the temporary build folder of the sketch:
Using previously compiled file: C:\Users\per\AppData\Local\Temp\arduino-sketch-97534792491BE5D1DEA89A9B820D7A08\libraries\SrcWrapper\HAL\stm32yyxx_hal_comp_ex.c.o
The cached core is in a separate folder (so it can be shared between multiple sketches):
Using precompiled core: C:\Users\per\AppData\Local\Temp\arduino-core-cache\core_ad5517dca5ea73b9e745dae1d6eed866.a
Delete the temporary build folder.
In the sample output I shared above, it is here:
The next time you compile the sketch, it will compile the libraries and core from source. Any changes you made to build_opt.h should take effect during that compilation.
If you know for certain the changes are only relevant to one specific cache, you can delete only that one in order to reduce the compilation time.
I'm strongly against adding any code for handling build_opt.h specifically. This is the sort of thing that accumulates over time making a codebase increasingly difficult to understand and maintain and prone to regressions. The deficiency should be solved in a generalized manner, not specific to a hack implemented by a 3rd party.
I should have written, not easily instead of not possible.
but as we say in france, it's "une usine à gaz" and it doesn't make you want to continue with the arduino idea, VSC does much better.
going to manually erase the caches is counterproductive.
the arduino ide is practical for beginners but not only them, you try to improve with version 2 but I have the impression that you refuse to give more options not to beginners, who will not remain so not long if he hangs the programming.
in my opinion, and it's only mine, it's a shame.
moreover displaying the detailed compilation increases by 5 or almost the compilation time because of a display bug still not corrected, that does not make you want either.
the output pauses for 5 seconds for each library line found, it's just impossible! (example Marlin, I compile it in 7mn with the IDE, and less than a minute with VSC).
I can make the effort to use VSC in this case, but for me it is "une usine à gaz" compared to your IDE.
if you don't want to associate patch code with build_opt.h and I understand it in terms of maintenance,
add at least one easily accessible option to empty this cache, without adding a command line type option, put a checkmark in preference and everyone will be happy.
I'm not sure where you got that impression. I have specifically stated in this thread that I support adding the ability to clear the cache. The need for such a thing has only recently emerged. We only have limited resources to dedicate to these incredibly complex open source projects. If the community wants to see a feature, maybe they should consider submitting a pull request to add it instead of expecting Arduino to do all the work?
Which version of the Arduino IDE are you using? This was a known bug with older versions:
But that bug was fixed months ago. We have not had a single report about the problem since then. It should not be surprising that a bug none of the developers experiences or has been notified of has not been fixed.
bodmer already submitted the request for this feature:
I am in support of it. It seems you are manufacturing a disagreement that doesn't exist.
we misunderstood each other, the translation is not correct to my turns of phrase.
I was just trying to help on a transparent way to handle this.
concerning the pauses in compilation output, indeed an old version not removed took over, since it works, with 2.0 like the last nightbuilds. but it's true that it gives the impression since there are plenty of 2 or 3 second pauses
I am all for anything that will EASILY allow the cache to be cleared. I don't believe beginners should have to be editing or invoking scripts. A simple pull-down menu option (in advanced or where ever) would be very much welcome. I am one of the people needing the build_opt.h for bodmer's TFT_eSPI library.
As far as compile times, the Arduino IDE has always been horribly slow with Windows setups. My RPi400 (Linux) compiles everything faster than my brand new (freshly installed) monster PC system I just built - which happens to be in the 99% range of all PCs on the planet when benchmarked. A sketch can take several minutes to compile the 2nd time (after cached - the fist time way longer) with this PC using the Arduino IDE (either 1.8x or 2.0.x) and my RPi400 takes 1/2 the time. Platform IO on the same PC is only typically 10-20 seconds for the 1st build depending on the complexity, and just a few seconds for cached builds. I have tried various different PCs and Windows versions all with the same result. I just chalked this up to this is how the Arduino IDE works with Windows. PlatformIO has its own quirks with Arduino/Espressif libraries so I would prefer to use the Aduino IDE 2.0.x for development but the compile times are a killer.
Exceptional compile times are usually found to be caused by interference by the antivirus. Most people use an antivirus on their Windows machines. On a Raspberry Pi, not so much.
Yes, I am aware of this. My development machine is not connected to the internet at all, and thus has absolutely no antivirus and firewalls installed, including Window's own. One would think that if there was an issue with an antivirus then it would affect PlatformIO or any of the Visual Studio stuff the same and that is not the case.
There was a time, long ago , when compile was almost instant, at least that was what it felt like compared to IDE 1. The problem then was ridicules long "Building sketch" and "indexing" times (not to mention startup load times). The latter was eventually solved but we did sacrifice some compile time - at least that was my experience.
I do not do very large sketches so the current compile times are quite acceptable to me. Oh, I am on Windows 10.
I am using Windows 10 (PRO) for my Windows setup, but I have tried Windows 8 and Windows 11 with the same results.
I have very large sketches with dozens of libraries being included. The real puzzling part is the fact that PlatformIO using the same Arduino libraries (imported project) is very fast, taking literally seconds to compile a large project once cached. We are talking many minutes of time difference here.
Compile time when using NO libraries at all is quite a bit faster than when including any small library with the Arduino IDE, but nowhere near as quick as PlatformIO. It seems the more libraries you add, the (substantially) longer the compile times get with the Arduino IDE.
, when compile was almost instant, at least that was what it felt like compared to IDE 1. The problem then was ridicules long "Building sketch" and "indexing" times (not to mention startup load times). The latter was eventually solved but we did sacrifice some compile time - at least that was my experience.