Upload via command line fails in script

Hello Community,

I have an Arduino connected to a Raspberry Pi. The RasPi runs a script that periodically checks an URL for an Arduino sketch and downloads it. The script then invokes the Arduino IDE via command line with the --upload option.
If I run the command myself, everything works as expected. But when the script calls the same command the compilation succeeds but it fails at the uploading stage. I have already provided the --verbose option, but that does not output anything useful for this particular problem :frowning:

I have attached the console outputs of both cases, fail.txt is the output produced by the script and success.txt is what I get when manually running the command.

It looks to me as though avrdude is not called, or if it is fails to start. I would expect at least an error message as output in the latter case though…
Does anyone have any ideas what might be causing this? The script is running as a system service and is executed as the root user, in case this might have something to do with it.

fail.txt (35.1 KB)

success.txt (38.7 KB)

Try replacing the contents of the script with a command that just runs avrdude directly, then execute that script exactly as you have been doing with the the one that runs the arduino command:

/opt/arduino-1.8.13/hardware/tools/avr/bin/avrdude

Do you still get an error?

Because there are no options or arguments, this command should just print out the command line help for AVRDUDE, which obviously is not terribly useful, but it acts as a minimal test of whether there is some fundamental problem that is preventing avrdude from being run under these conditions, or whether the issue is specific to the Arduino IDE's attempt to run the avrdude command. If the former, it may provide a useful error message that will give a better clue than that frustratingly vague "An error occurred while uploading the sketch" from the Arduino IDE.

Hi pert, I did what you suggested. I replaced the arduino upload command with the raw avrdude and ran the script. The script then printed the help of avrdude, as expected, and without any errors, at least that I can see…

Usage: avrdude [options]
Options:
  -p <partno>                Required. Specify AVR device.
  -b <baudrate>              Override RS-232 baud rate.
  -B <bitclock>              Specify JTAG/STK500v2 bit clock period (us).
  -C <config-file>           Specify location of configuration file.
  -c <programmer>            Specify programmer type.
  -D                         Disable auto erase for flash memory
  -i <delay>                 ISP Clock Delay [in microseconds]
  -P <port>                  Specify connection port.
  -F                         Override invalid signature check.
  -e                         Perform a chip erase.
  -O                         Perform RC oscillator calibration (see AVR053). 
  -U <memtype>:r|w|v:<filename>[:format]
                             Memory operation specification.
                             Multiple -U options are allowed, each request
                             is performed in the order specified.
  -n                         Do not write anything to the device.
  -V                         Do not verify.
  -u                         Disable safemode, default when running from a script.
  -s                         Silent safemode operation, will not ask you if
                             fuses should be changed back.
  -t                         Enter terminal mode.
  -E <exitspec>[,<exitspec>] List programmer exit specifications.
  -x <extended_param>        Pass <extended_param> to programmer.
  -y                         Count # erase cycles in EEPROM.
  -Y <number>                Initialize erase cycle # in EEPROM.
  -v                         Verbose output. -v -v for more.
  -q                         Quell progress output. -q -q for less.
  -l logfile                 Use logfile rather than stderr for diagnostics.
  -?                         Display this usage.

avrdude version 6.3-20190619, URL: <http://savannah.nongnu.org/projects/avrdude/>

To me, this suggests that avrdude can in fact run correctly but for whatever reason just doesn’t.

But I think I can do a workaround: Something I forgot to mention is, as part of the download/upload to Arduino routine I also call the Arduino CLI with the --verify option and that seems to work without problems. I know there is the option to keep the temporary files that are created. I guess I could invoke avrdude directly and pass those files as parameters, couldn’t I? I think I’ve seen the invocation, or more specifically the required parameters, in the verbose log. Can I just copy-paste that or do I need to do anything else (cd into the directory or whatever)?

Thanks fo the help so far.

clemens_baudisch:
To me, this suggests that avrdude can in fact run correctly but for whatever reason just doesn't.

It does narrow the issue down to being Arduino IDE-specific.

clemens_baudisch:
I think I've seen the invocation, or more specifically the required parameters, in the verbose log.

Here it is from the "success.txt" you posted earlier:

/opt/arduino-1.8.13/hardware/tools/avr/bin/avrdude -C/opt/arduino-1.8.13/hardware/tools/avr/etc/avrdude.conf -v -patmega328p -carduino -P/dev/ttyUSB0 -b115200 -D -Uflash:w:/tmp/arduino_build_452727/job.ino.hex:i

So you would only need to modify the path to the .hex file to point to the custom directory you set via the Arduino IDE command line option.

clemens_baudisch:
Can I just copy-paste that or do I need to do anything else (cd into the directory or whatever)?

In the case of this specific board, nothing special is required. Some of the other Arduino boards are a little more tricky to upload to, but it's simple with the ATmega328P board like Uno or Nano. You could have run that same avrdude command from the command line and get the same results.


Another thing you could try is using Arduino CLI instead of the Arduino IDE's CLI.
https://arduino.github.io/arduino-cli/latest/installation/
Arduino created Arduino CLI with this very sort of automated application in mind. Because I don't understand the cause of the upload process failure with the Arduino IDE CLI, I don't have any specific reason to believe the same won't occur when using Arduino CLI, but it is possible. Even if the upload still fails, you might enjoy using Arduino CLI for the sketch compilation, then still run the avrdude command directly as you plan.

I just tried it and it worked like a charm. Thank you. It also has the positive side effect that it speeds up the script execution time since I skip the second time compiling the sketch for the upload.

I also think I may have an idea what was wrong in the first place. After my script successfully uploaded my sketch I tried to open the serial port the Arduino is on with a terminal program to see the Arduino's output and I got an error. I also noticed that, having left it alone for a few days because I had other projects I needed to work on, the original version of my script (the one with bot arduino calls) did succeed in uploading a sketch, but only once.
I suspect that something may be blocking the serial port and arduino just gives the somewhat cryptic "failed to upload sketch" message. But why avrdude on it's own worked in this scenario is beyond me, since it also needs the serial port and should fail if the port is blocked. Unless it is the something that is blocking the port...

I may post updates if I find out anything noteworthy for anyone who might have a similar problem in the future, but otherwise I consider the problem solved, even if it is in a roundabout way.

You're welcome. I'm glad to hear it's working now. I think that even the compilation will be a bit faster when using Arduino CLI due to it not having as much start up overhead.

Thanks for taking the time to post an update with your findings.

Enjoy!
Per