What does path to avrdude being __REMOVED__ mean?

Development Tools IDE 1.x
1

I know the usual tricks to fix this; these will not work because I am hacking up an installation with locally present tools and compiler so I can develop these things on one system without fucking them up system-wide by using board manager installation.

This is the error to which I refer:

___REMOVE___/bin/avrdude -CC:\Users\Pete\Documents\Arduino\hardware\megaTinyCore\megaavr/avrdude.conf -v -pattiny3216 -cjtag2updi -PCOM3 -Uflash:w:C:\Users\Pete\AppData\Local\Temp\arduino_build_253941/ws2812test.ino.hex:i 
java.io.IOException: Cannot run program "___REMOVE___/bin/avrdude": CreateProcess error=2, The system cannot find the file specified
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at processing.app.helpers.ProcessUtils.exec(ProcessUtils.java:26)
	at cc.arduino.packages.Uploader.executeUploadCommand(Uploader.java:129)
	at cc.arduino.packages.uploaders.SerialUploader.runCommand(SerialUploader.java:383)
	at cc.arduino.packages.uploaders.SerialUploader.uploadUsingProgrammer(SerialUploader.java:315)
	at cc.arduino.packages.uploaders.SerialUploader.uploadUsingPreferences(SerialUploader.java:90)
	at cc.arduino.UploaderUtils.upload(UploaderUtils.java:77)
	at processing.app.SketchController.upload(SketchController.java:732)
	at processing.app.SketchController.exportApplet(SketchController.java:703)
	at processing.app.Editor$UploadHandler.run(Editor.java:2055)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
	at java.lang.ProcessImpl.create(Native Method)
	at java.lang.ProcessImpl.<init>(ProcessImpl.java:386)
	at java.lang.ProcessImpl.start(ProcessImpl.java:137)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 10 more
An error occurred while uploading the sketch

Unfortunately, in very unpredictable circumstances, (I have multiple installations where I thought I did everything the same, but one version works, and the other version gives the REMOVED error, and I don't know how to make it not do that. The "reinstall everything" strategy here is no good, because if I do that, the problem just comes back when I put my mods in)

2

This happens when the platform's tool dependency is missing. The REMOVE is a placeholder that's supposed to be replaced by the real path once the tool dependency resolution process is finished, but if the process fails then the placeholder is not replaced and you get this confusing error message, certainly not an ideal user experience.

With avrdude, if you haven't specified a version in platform.txt, it should normally be able to get avrdude from either the Arduino IDE's bundled Arduino AVR Boards platform, or from Arduino15/packages/arduino/tools/avrdude, so as long it is installed at one of those locations you shouldn't get the REMOVE error. However, it sounds like your setup is not the standard installation, so that might not be so in this particular case.

3

Aaahaahaaa!

Thank you - that's what I needed!

So it was a virgin nightly in portable mode, but with sketchbook set to my normal sketchbook folder (to pick up latest cores as manual installs), except that two things were changed:

my hacked up tools folder from my butchered 1.8.9 with modified (see Auto-get atpacks, get+process DA-series atpack, files for recent issues (but see within - I don't know how to make it use them) by SpenceKonde · Pull Request #68 · arduino/toolchain-avr · GitHub - I have no idea how to patch the files, and lost interest once I'd made packages from the arduino5 binaries and packages, and support files I provided, such that I could release my core - at least for preliminary testing - if I needed to - by hosting my modified compiler packages)
I also put the arduino17 avrdude binary over the old one.... but what I didn't do is change the builtin_tools_versions.txt line for avrdude. But the newer arduino core now required 17, looked at that, ans it's package_index_bundled.json and dependency resolution failed.

Interestingly enough, I had changed the name of the avrgcc version in builtin_tools_versions.txt to arduino.avr-gcc=7.3.0-atmel3.6.1-azzyfix1 - that didn't end up with a REMOVE ... is it only avrdude that does that?

Even besides this problem, I am so glad I understand this now!

Now instead of a mystery problem with a brute force solution, it's something I understand (for which to a novice, the brute force solution is probably a good one - though wouldn't upgrading and/or downgrading appropriate board package via board manager be likely to fix it? It's the sort of thing where one really wonders how it winds up happening in the field, and (present case excepted - though I knew I was doing crazy stuff) always to newbies...

And yeah, that is an awful user experience... It can't be hard to check the path for "REMOVED", catch the error, and print out a helpful message "avrdude version 6.3.0-arduino17 does not appear to be installed??? Try upgrading (name of package that it got the requirement from) from Tools -> Boards Manager. If the latest version is already installed, go downgrade one version and then upgrade again. (assuming that's expected to fix it) If this does not work, follow the instructions at (URL describing a clean reinstall procedure)"

Heck, even just changing the placeholder to something more descriptive than REMOVED would help!

Arduino is a strange juxtaposition of intense handholding at great expense in the API and no hand-holding at all in the event of certain errors.

4

DrAzzy:
that didn't end up with a REMOVE ... is it only avrdude that does that?

It's not only avrdude. I can get the same by removing Arduino15/packages/arduino/tools/bossac and then trying to upload to a SAMD board. So I think it happens for any missing upload tool.

However, in my quick experiments I wasn't able to get the REMOVE error by removing compiler tool (I just get an 'exec: "/bin/arm-none-eabi-g++": file does not exist' error). I don't understand why there is a difference between the error for the upload tool and the compiler tool. I'd expect the same system to be used for all tools.

DrAzzy:
though wouldn't upgrading and/or downgrading appropriate board package via board manager be likely to fix it?

My experience is that the IDE isn't very good at recovering from the common occurrence of this type of problem (when something goes wrong during a package installation and the user ends up with an empty directory where there shouldn't be one), so just nuking the vendor folder (or the whole Arduino15 folder if you want to make it simple, but slow, for the user), is the sure fire way to fix it.

With my contrived setup to generate this error, when I try installing another version of the package, or removing it, with Boards Manager, I just get an NPE:

java.lang.NullPointerException
java.lang.RuntimeException: java.lang.NullPointerException
	at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onInstallPressed$2(ContributionManagerUI.java:175)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
	at cc.arduino.contributions.packages.ContributionInstaller.remove(ContributionInstaller.java:276)
	at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onInstallPressed$2(ContributionManagerUI.java:170)
	... 1 more

and the package is unaffected so the issue remains.

DrAzzy:
And yeah, that is an awful user experience... It can't be hard to check the path for "REMOVED", catch the error, and print out a helpful message "avrdude version 6.3.0-arduino17 does not appear to be installed??? Try upgrading (name of package that it got the requirement from) from Tools -> Boards Manager. If the latest version is already installed, go downgrade one version and then upgrade again. (assuming that's expected to fix it) If this does not work, follow the instructions at (URL describing a clean reinstall procedure)"

I think part of the problem is that the developers simply don't hit this situation. It's not common at all for it to occur by accident. I think there was one IDE version like 1.6.6 or something that had a bug that caused it, but I haven't had it since then other than when I intentionally make it happen. When you have a million users banging away at some software, then that rare corner case is going to pop up periodically, but we usually deal with that here on the forum, where the developers don't see it.

Another thing is that when you are intimately familiar with some software, you are going to instinctively use it in the manner you designed it for. The developers automatically put the tools dependencies in the right places when they are testing. It's only when you have some random user give the software a try that all those use cases you never considered come out. I had an experience like this last week. I've been working on a CI tool for compilation testing of Arduino projects. One of Arduino's firmware developers and I have been using this thing for months without finding any problems. Then one of the other developers gave it a try and immediately attempted a usage I had never considered, which was not working due to a bug and was also a poor user experience due to lack of documentation.