How do I use the latest arduino-cli with the current IDE?

Development Tools IDE 2.x
1

I wanted to test the fix for avoiding unnecessary recompiles in the IDE, which I had found here: Do not recompile the sketch if there are no changes and libraries have not changed · Issue #2204 · arduino/arduino-cli · GitHub

It is suggested that I use the latest cli, which I've downloaded.
I'm on macOS, and the cli is named "arduino-cli".

However, I cannot find a similarly named file in the IDE nor anywhere else on the Mac that I'd replace to use the new version, so I can't figure out how to test the new cli. I'd think the CLI is invoked by the IDE as a tool. Isn't it? (Update: It's indeed inside the app, I was just looking in the wrong place)

If the CLI is not a separate tool within the IDE, then I guess its code is directly compiled into the IDE somewhere. In that case, how do I test this? I have no idea how to use the CLI; I only use the IDE, and I'm not eager to spend hours on reading docs only to run this quick test. Is there an easy way to find out the necessary cmdline args that the IDE would use with the CLI tool?

2

Welcome to the forum

Which version of the IDE are you using ?

3

Sorry, is that really important? Do you know that some versions of the IDE are including the CLI while others don't? You did read the linked issue, yes? The one where we talk about using nightly builds?

4

I am on a Mac (MBP M5) and if nothing changes the re-compile goes very fast. Unfortunatly it appears it doesn't take much to cause a change and I do not know if it is a Mac specific issue.

The download is available HERE

It looks like this, click the 64 bit

Screenshot 2026-04-27 at 11.21.35
Screenshot 2026-04-27 at 11.21.351750×1182 189 KB

and the download looks like this

Screenshot 2026-04-27 at 11.23.40
Screenshot 2026-04-27 at 11.23.40636×154 5.06 KB

5

I just did a crude test using the IDE. The amount of compiler log file output is MUCH longer than a second compile. I can't tell you by how much but to the best of my knowledge that is how the IDE works and this crude test at least backs that up.

6

I've already managed to download the file; I thought I explained that?

The question is where to place it so that the IDE uses it instead of the one the IDE came with. This is under the assumption that the IDE uses an external cli tool for building the programs, and I'm asking for clarification about this.

The instructions in your image say that I just have to place it into a PATH location, but since which arduino-cli leads to arduino-cli not found, this suggests to me that this isn't true. And, alas, if I place the downloaded arduino-cli into a dir my PATH contains, and then restart the IDE, the "Verify" still takes 10 seconds every time "compiling", which is still not right.

So, that seems to prove that I cannot simply use the new cli with an existing IDE, and I am asking if there's way to do that.

7

What is "very fast" for you? If you've never seen anything else you might feel that 5 seconds is very fast.

The issue is this: A second "Verify" should not take seconds, it should take milliseconds, if none of the source files were changed. Basically, that should involve only a few checks with the file system, to see if any of the source files have a changed modification date. If not, there were no changes and nothing else should happen. That's how practically any other build system works, and the issue with this Arduino system is that this isn't done right, for hard-to-find reasons. I thought that the quoted cli fix should deal with that, and wanted to verify it, or provide feedback if it still isn't doing it right.

8

I went into terminal and typed SET. I then saw that the scripts folder I created a while back in my home folder was in the path so I dropped the arduino-cli there. I then in terminal typed arduino-cli and got this

Screenshot 2026-04-27 at 11.39.12
Screenshot 2026-04-27 at 11.39.121920×1078 222 KB

9

I am no expert, but my limited understanding of the 'verify' (compile) process for the Arduino eco-system is vastly different than what I used with make several decades ago and I am guessing what you are used to. The way that the arduino engine resolves library dependencies is somewhat bizarre and even backwards from my professional experience but that is now over 20 years ago.

If I may be so blunt, you are applying expectations based on one technology onto a different technology. I think the verify process is described in the docs but I have no idea where other than staring at www.arduino.cc

Good luck.

10

I appreciate your effort, but I know how do do this. I'm no beginner, in case you thought.

11

Just an FYI, here is the 2nd compile I did, the first produced 252 lines the second only 53. If you look at those 53 maybe you will get an idea why it still takes seconds as opposed to ms.
At line 32 in both runs it says 'Compiling core'. The second run then at line 34 says 'Using pre-compiled core' whereas the first run took from line 33 to 233 to compile the core.
That's all I know, I will leave it to the experts to answer your next question.

loading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/230-Ultimate-Switch-Bounce-Eliminator: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/ESP32-Cheap-Yellow-Display: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/EspExceptionDecoder: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/FreeRTOS-Kernel-Book: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/KAMP-for-K1-Series: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/MiniCore: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/PN532: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/SDVirtualRAM-main: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/ServoSmoothing: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/Solar-Power-Load-Sharing: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/TTGO-T-Display: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/TeensyDocuments: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/XIAO-PowerBread: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/artyom-beilis.github.io: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/docs: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/electronics_cookbook: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/experiment-with-the-sense-hat: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/led_task: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/nebula_firmware: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/pi-spi-display: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/pure-bash-bible: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/pure-sh-bible: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/simple-electronics-gpiozero-2e: invalid library: no header files foundloading library from /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/zephyr: invalid library: no header files found
FQBN: esp32:esp32:XIAO_ESP32S3
Using board 'XIAO_ESP32S3' from platform in folder: /Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8
Using core 'esp32' from platform in folder: /Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8

/usr/bin/env bash -c "[ ! -f \"/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy\"/partitions.csv ] || cp -f \"/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy\"/partitions.csv \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/partitions.csv"
/usr/bin/env bash -c "[ -f \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/partitions.csv ] || [ ! -f \"/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/variants/XIAO_ESP32S3\"/partitions.csv ] || cp \"/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/variants/XIAO_ESP32S3\"/partitions.csv \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/partitions.csv"
/usr/bin/env bash -c "[ -f \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/partitions.csv ] || cp \"/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8\"/tools/partitions/default_8MB.csv \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/partitions.csv"
/usr/bin/env bash -c "[ -f \"/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy\"/bootloader.bin ] && cp -f \"/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy\"/bootloader.bin \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/Dummy.ino.bootloader.bin || ( [ -f \"/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/variants/XIAO_ESP32S3\"/bootloader.bin ] && cp \"/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/variants/XIAO_ESP32S3\"/bootloader.bin \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/Dummy.ino.bootloader.bin || \"/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esptool_py/5.2.0\"/esptool --chip esp32s3 elf2image --flash-mode dio --flash-freq 80m --flash-size 8MB -o \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/Dummy.ino.bootloader.bin \"/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8\"/bin/bootloader_qio_80m.elf )"
esptool v5.2.0
Creating ESP32-S3 image...
Merged 2 ELF sections.
Successfully created ESP32-S3 image.
/usr/bin/env bash -c "[ ! -f \"/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy\"/build_opt.h ] || cp -f \"/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy\"/build_opt.h \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/build_opt.h"
/usr/bin/env bash -c "[ -f \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/build_opt.h ] || : > \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/build_opt.h"
/usr/bin/env bash -c ": > '/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/file_opts'"
/usr/bin/env bash -c "cp -f \"/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8\"/sdkconfig \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/sdkconfig"
Detecting libraries used...
/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp-x32/2601/bin/xtensa-esp32s3-elf-g++ -c @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/cpp_flags -w -Os -Werror=return-type -w -x c++ -E -CC -DF_CPU=240000000L -DARDUINO=10607 -DARDUINO_XIAO_ESP32S3 -DARDUINO_ARCH_ESP32 -DARDUINO_BOARD="XIAO_ESP32S3" -DARDUINO_VARIANT="XIAO_ESP32S3" -DARDUINO_PARTITION_default_8MB -DARDUINO_HOST_OS="macosx" -DARDUINO_FQBN="esp32:esp32:XIAO_ESP32S3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=default,MSCOnBoot=default,DFUOnBoot=default,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=8M,PartitionScheme=default_8MB,DebugLevel=none,PSRAM=disabled,LoopCore=1,EventsCore=1,EraseFlash=none,JTAGAdapter=default" -DESP32=ESP32 -DCORE_DEBUG_LEVEL=0 -DARDUINO_RUNNING_CORE=1 -DARDUINO_EVENT_RUNNING_CORE=1 -DARDUINO_USB_MODE=1 -DARDUINO_USB_CDC_ON_BOOT=1 -DARDUINO_USB_MSC_ON_BOOT=0 -DARDUINO_USB_DFU_ON_BOOT=0 @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/defines -I/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy -iprefix /Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/include/ @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/includes -I/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/qio_qspi/include -I/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/cores/esp32 -I/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/variants/XIAO_ESP32S3 @/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/build_opt.h @/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/file_opts /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/sketch/Dummy.ino.cpp -o /dev/null
Alternatives for debug.h: [debug]
ResolveLibrary(debug.h)
  -> candidates: [debug]
/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp-x32/2601/bin/xtensa-esp32s3-elf-g++ -c @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/cpp_flags -w -Os -Werror=return-type -w -x c++ -E -CC -DF_CPU=240000000L -DARDUINO=10607 -DARDUINO_XIAO_ESP32S3 -DARDUINO_ARCH_ESP32 -DARDUINO_BOARD="XIAO_ESP32S3" -DARDUINO_VARIANT="XIAO_ESP32S3" -DARDUINO_PARTITION_default_8MB -DARDUINO_HOST_OS="macosx" -DARDUINO_FQBN="esp32:esp32:XIAO_ESP32S3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=default,MSCOnBoot=default,DFUOnBoot=default,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=8M,PartitionScheme=default_8MB,DebugLevel=none,PSRAM=disabled,LoopCore=1,EventsCore=1,EraseFlash=none,JTAGAdapter=default" -DESP32=ESP32 -DCORE_DEBUG_LEVEL=0 -DARDUINO_RUNNING_CORE=1 -DARDUINO_EVENT_RUNNING_CORE=1 -DARDUINO_USB_MODE=1 -DARDUINO_USB_CDC_ON_BOOT=1 -DARDUINO_USB_MSC_ON_BOOT=0 -DARDUINO_USB_DFU_ON_BOOT=0 @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/defines -I/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy -iprefix /Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/include/ @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/includes -I/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/qio_qspi/include -I/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/cores/esp32 -I/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/variants/XIAO_ESP32S3 -I/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/debug @/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/build_opt.h @/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/file_opts /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/sketch/Dummy.ino.cpp -o /dev/null
Generating function prototypes...
/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp-x32/2601/bin/xtensa-esp32s3-elf-g++ -c @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/cpp_flags -w -Os -Werror=return-type -w -x c++ -E -CC -DF_CPU=240000000L -DARDUINO=10607 -DARDUINO_XIAO_ESP32S3 -DARDUINO_ARCH_ESP32 -DARDUINO_BOARD="XIAO_ESP32S3" -DARDUINO_VARIANT="XIAO_ESP32S3" -DARDUINO_PARTITION_default_8MB -DARDUINO_HOST_OS="macosx" -DARDUINO_FQBN="esp32:esp32:XIAO_ESP32S3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=default,MSCOnBoot=default,DFUOnBoot=default,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=8M,PartitionScheme=default_8MB,DebugLevel=none,PSRAM=disabled,LoopCore=1,EventsCore=1,EraseFlash=none,JTAGAdapter=default" -DESP32=ESP32 -DCORE_DEBUG_LEVEL=0 -DARDUINO_RUNNING_CORE=1 -DARDUINO_EVENT_RUNNING_CORE=1 -DARDUINO_USB_MODE=1 -DARDUINO_USB_CDC_ON_BOOT=1 -DARDUINO_USB_MSC_ON_BOOT=0 -DARDUINO_USB_DFU_ON_BOOT=0 @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/defines -I/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy -iprefix /Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/include/ @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/includes -I/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/qio_qspi/include -I/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/cores/esp32 -I/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/variants/XIAO_ESP32S3 -I/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/debug @/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/build_opt.h @/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/file_opts /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/sketch/Dummy.ino.cpp -o /private/var/folders/g9/g923bvbn7995gtv9ndrdbkdc0000gn/T/1680478734/sketch_merged.cpp
/Users/ronalexander/Library/Arduino15/packages/builtin/tools/ctags/5.8-arduino11/ctags -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives /private/var/folders/g9/g923bvbn7995gtv9ndrdbkdc0000gn/T/1680478734/sketch_merged.cpp

Compiling sketch...
/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp-x32/2601/bin/xtensa-esp32s3-elf-g++ -MMD -c @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/cpp_flags -Wall -Wextra -Os -Werror=return-type -DF_CPU=240000000L -DARDUINO=10607 -DARDUINO_XIAO_ESP32S3 -DARDUINO_ARCH_ESP32 "-DARDUINO_BOARD=\"XIAO_ESP32S3\"" "-DARDUINO_VARIANT=\"XIAO_ESP32S3\"" -DARDUINO_PARTITION_default_8MB "-DARDUINO_HOST_OS=\"macosx\"" "-DARDUINO_FQBN=\"esp32:esp32:XIAO_ESP32S3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=default,MSCOnBoot=default,DFUOnBoot=default,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=8M,PartitionScheme=default_8MB,DebugLevel=none,PSRAM=disabled,LoopCore=1,EventsCore=1,EraseFlash=none,JTAGAdapter=default\"" -DESP32=ESP32 -DCORE_DEBUG_LEVEL=0 -DARDUINO_RUNNING_CORE=1 -DARDUINO_EVENT_RUNNING_CORE=1 -DARDUINO_USB_MODE=1 -DARDUINO_USB_CDC_ON_BOOT=1 -DARDUINO_USB_MSC_ON_BOOT=0 -DARDUINO_USB_DFU_ON_BOOT=0 @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/defines -I/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy -iprefix /Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/include/ @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/includes -I/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/qio_qspi/include -I/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/cores/esp32 -I/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/variants/XIAO_ESP32S3 -I/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/debug @/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/build_opt.h @/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/file_opts /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/sketch/Dummy.ino.cpp -o /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/sketch/Dummy.ino.cpp.o
Compiling libraries...
Compiling library "debug"
Compiling core...
/usr/bin/env bash -c "echo -DARDUINO_CORE_BUILD > '/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/file_opts'"
Using precompiled core: /Users/ronalexander/Library/Caches/arduino/cores/23263dd6aafe2042c45c10f0c9717bf0/core.a
/usr/bin/env bash -c ": > '/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/file_opts'"
Linking everything together...
/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp-x32/2601/bin/xtensa-esp32s3-elf-g++ -Wl,--Map=/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.map -L/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/lib -L/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/ld -L/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/qio_qspi -Wl,--wrap=esp_panic_handler @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/ld_flags @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/ld_scripts -Wl,--start-group /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/sketch/Dummy.ino.cpp.o /Users/ronalexander/Library/Caches/arduino/cores/23263dd6aafe2042c45c10f0c9717bf0/core.a @/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8/flags/ld_libs -Wl,--end-group -Wl,-EL -o /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.elf
/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esptool_py/5.2.0/esptool --chip esp32s3 elf2image --flash-mode dio --flash-freq 80m --flash-size 8MB --elf-sha256-offset 0xb0 -o /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.bin /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.elf
esptool v5.2.0
Creating ESP32-S3 image...
Merged 2 ELF sections.
Successfully created ESP32-S3 image.
python3 /Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/tools/gen_esp32part.py -q /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/partitions.csv /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.partitions.bin
/usr/bin/env bash -c "[ ! -d \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/libraries/Insights ] || python3 \"/Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8\"/tools/gen_insights_package.py /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2 Dummy.ino \"/Users/ronalexander/Documents/Electronics/SourceCode/Arduino/5-Models/Dummies/Dummy\""
/usr/bin/env bash -c "[ ! -d \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/libraries/ESP_SR ] || [ ! -f \"/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8\"/esp_sr/srmodels.bin ] || cp -f \"/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp32s3-libs/3.3.8\"/esp_sr/srmodels.bin \"/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2\"/srmodels.bin"
/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esptool_py/5.2.0/esptool --chip esp32s3 merge-bin -o /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.merged.bin --pad-to-size 8MB --flash-mode keep --flash-freq keep --flash-size keep 0x0 /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.bootloader.bin 0x8000 /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.partitions.bin 0xe000 /Users/ronalexander/Library/Arduino15/packages/esp32/hardware/esp32/3.3.8/tools/partitions/boot_app0.bin 0x10000 /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.bin
esptool v5.2.0
Wrote 0x800000 bytes to file '/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.merged.bin', ready to flash to offset 0x0.
/usr/bin/env bash -c "echo '--flash-mode dio --flash-freq 80m --flash-size 8MB'>'/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/flash_args'&& echo '0x0 Dummy.ino.bootloader.bin'>> '/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/flash_args'&& echo '0x8000 Dummy.ino.partitions.bin'>> '/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/flash_args'&& echo '0xe000 boot_app0.bin'>> '/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/flash_args'&& echo '0x10000 Dummy.ino.bin'>> '/Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/flash_args'"
Using library debug in folder: /Users/ronalexander/Documents/Electronics/SourceCode/Arduino/libraries/debug (legacy)
/Users/ronalexander/Library/Arduino15/packages/esp32/tools/esp-x32/2601/bin/xtensa-esp32s3-elf-size -A /Users/ronalexander/Library/Caches/arduino/sketches/5746FECD041220B97FA60757585921F2/Dummy.ino.elf
Sketch uses 285635 bytes (8%) of program storage space. Maximum is 3342336 bytes.
Global variables use 21636 bytes (6%) of dynamic memory, leaving 306044 bytes for local variables. Maximum is 327680 bytes.
12

Okay, thank you for expanding on your point of view. I gave some background on this in the previous comment (Do not recompile the sketch if there are no changes and libraries have not changed · Issue #2204 · arduino/arduino-cli · GitHub). Basically, any build system has to decide somehow if the source files need compiling, and any logic says that if you have not changed the source files, then there should be no need to recompile anything. The explanations in some tickets that mention complication with library detection are secondary - there's no need to re-scan for libs if the sources are unchanged. Simple as that. If the devs have trouble keeping track of what the sources are and whether they have changed, that's something I'd like to see if I can help with.

13

I can tell you have some experience. Just FYI, first computer from Ferranti Packard 1959 as a gift to my High School, now I am 84 and shuttered my Consulting business in 2004 after letting two employees go.

14

I think you mis-spoke, the libs have to be checked for changes as well don't they?
I see one of the experts is replying so I will bid you adieu.

15

Oh, you beat me! I am only 61, and started in 1980 with a simple 6502 board with hex key input :slight_smile:

16

This makes it very challenging to replace the Arduino CLI that is bundled with the Arduino IDE installation. macOS has security restrictions to prevent any modifications to an app's installation.

So I recommend you don't do this. My understanding is that your goal was to evaluate the effect of the resolution to Do not recompile the sketch if there are no changes and libraries have not changed · Issue #2204 · arduino/arduino-cli · GitHub. The most efficient way to do that will be to simply use Arduino CLI directly from the terminal. We are happy to provide assistance in doing that if you need it. You can get an overview here:

https://arduino.github.io/arduino-cli/latest/getting-started/

If you do choose to attempt to modify the Arduino IDE app, you will need to replace the executable file that is located at this path under the Arduino IDE app:

<Arduino IDE app>/Contents/Resources/app/lib/backend/resources/arduino-cli

(where <Arduino IDE app> is the location of the Arduino IDE app on your hard drive)

Correct.

You only need to compile the sketch to test the changes. The equivalent of Arduino IDE's "Verify/Compile" is the arduino-cli compile command. It is documented here:

https://arduino.github.io/arduino-cli/latest/commands/arduino-cli_compile/

It can be in some cases. The Arduino CLI developers periodically change the interface through which Arduino IDE communicates with Arduino CLI. If there were breaking changes to that interface between the version of Arduino CLI that was bundled with the Arduino IDE installation and the version of Arduino CLI you replace it with, then that can result in Arduino IDE not working correctly. However, the Arduino CLI API is now fairly stable so in practice since you are using the latest version of Arduino IDE this isn't likely to be a problem.

Arduino IDE will not use the Arduino CLI installation from PATH. It always uses the Arduino CLI that is bundled with the app.

17

Well…

The build directory already contains extra files mentioning which libs are used by the project. Again, those also just consist of files that need to be checked for changes. What appears to be missing right now (as with IED 2.3.8) is that there's no fine grained map of all involved source and their output files. If that were there, it would still allow a quick check of all involved file dates to decide if anything needs rebuilding.

So I wonder if the real problem is that the tools that are used right now do not export the dependency information in a way that the higher level task (the one that decides what needs rebuilding) can parse sufficiently.

18

Hi Per,

thank you for taking the time to answer.

  1. I am a macOS dev, I can codesign and notarize if necessary.
  2. I was making a stupid mistake: When I searched for the cli file, I overlooked that I had placed the IDE.app onto a separate volume, and I only searched the startup volume. The really embarassing part of this is that I'm the author of a leading file find program for the Mac, and I should've known better!

I'll test it now.

19

OK, great, then not so difficult as I had implied. Then it may indeed be as easy for you to use Arduino CLI via Arduino IDE by replacing the bundled arduino-cli executable than it would be for you to run the appropriate arduino-cli compile command from the terminal. However, the offer to provide assistance with the latter still stands.

20

Okay, here's what I found:

  1. With the new cli, a "Verify" on an unchanged project seems to be maybe 20-30% faster. Where it took 5s before, it takes more like 2.5 to 3s now, roughly. So, that's a nice improvement.
  2. Using the CLI directly isn't as easy as reading the help page about it, because I have no idea what board (-b option) I need to put there, nor which other options may need passing to get the same as with the IDE's invocation. In that regard, it would be nice if the Output view would include the tools and their args when they're called, at least optionally (I'd hope for a pref setting). Or maybe there's a log file that contains all that already?

Regardless, I am still puzzled why it takes that long. As I explained above, the checks of the modification dates of multiple files, even if it's a 1000 files, shouldn't take even a second. So, clearly, the caching is still incomplete and I like to understand what the challenges are. See my previous post #12.

I wouldn't usually get involved in something like this, but since I'm currently using the IDE every day for a larger project I'm working on, I'm really getting to the point where I want to see if I can help.