Update Arduino without the IDE

Is it possible to compile the sketch in the IDE and have it write a binary file that you can take on a USB stick for example to another computer and upload it to the Arduino?

Yes. Just do a verify operation, and it will leave the files used in the compilation in the temporary directory, which varies by OS. On my Linux system, it creates a new directory /tmp/build*.tmp, where it copies all of the files used in the compilation, munges the .ino/.pde file, and then builds it. It leaves a file with the .hex extension. You would need to dig into avrdude to figure out what is the appropriate line to send the hex file to the target.

When I was trying to program my trinket/gemma, I couldn't use the IDE to do the download, but instead needed to use avrdude to load the program. When I was doing this, I did a rm -rf /tmp/build*.tmp /tmp/untitled*.tmp before running the arduino program. I would only compile one sketch, and from the command line, I could find it via:
$ ls /tmp/build*.tmp/*.hex

If you turn on verbose compiling you find the exact pathname of the .hex file. It is usually one of the last lines of the compile.

eg.

/var/folders/1l/43x8v10s1v36trvjz3v92m900000gn/T/build8678149160504517695.tmp/sketch_oct29d.cpp.elf
/var/folders/1l/43x8v10s1v36trvjz3v92m900000gn/T/build8678149160504517695.tmp/sketch_oct29d.cpp.hex
Binary sketch size: 4,352 bytes (of a 32,256 byte maximum)

Thanks for the info guys. I found my .hex file. So very interesting. I always assumed it was some sort of binary file like a compiled .exe but it's just an ascii text file with hex inside of it. Not to get side tracked but is it possible to take that hex file and convert back to a sketch? or is that a one way conversion?

So now that I have my hex file I need to figure out avrdude and it looks like I'll be set.

You can get the assembler code back, but no, you can't get your sketch back.

The hex file is a representation of the program file. The IDE first creates the .elf file, which is the binary format used by the compiler (in the past, there were other object file formats, such as A.OUT, B.OUT, COFF, etc. but ELF is the main object file format put out by the embedded GCC targets and the Linux system).

After making the ELF file, the IDE uses the objdump program to convert the ELF file into HEX, which as you noticed is ascii text. The reason is serial drivers often times used in-band signaling for flow control to slow down the transmission if the receiver can't keep up, using control-S to stop the transmission, and control-Q to re-enable it. However the problem is if you are tranmitting a binary value, it very well might be control-S or control-Q. In addition, tabs might be converted into spaces, trailing whitespace might be removed, and a single newline might be converted into carriage return and newline. On real serial lines using DB-9 cables or the earlier 25 pin cable, you had additional lines that the receiver could enable to turn on hardware flow control.

So, the download format was made into a textual representation, using 1-3 printable characters for each byte in the program. This way, no matter whether software flow control or hardware flow control was used, and whether tabs/newlines were munged, the program would still be able to be installed. You could even print the hex file output and put it in a book, and somebody could type in the characters to recreate the hex file.

Back in the day, when I worked on embedded GCC targets, I recall one target board, could not do either software or hardware flow control, and I needed to do the download by printing the line, and putting in a delay so the board would not lose data. Of course this was back in the day, when computers had real RS-232 serial ports and parallel ports as well.

There are a couple of main formats. Arduino uses Intel Hex (Intel HEX - Wikipedia). A competing format was Motorola SREC or S-records (SREC (file format) - Wikipedia). There are other formats, and here is a comparison: Binary-to-text encoding - Wikipedia.

If you turn on verbose compiling you find the exact pathname of the .hex file.

You can also ferret it out (under Windows) with a simple mod to my "elf" script finder.
http://forum.arduino.cc/index.php?action=profile;sa=showPosts
(Link above tested)

(Untested Edit)

PATH=%path%;C:\Program Files\Arduino_105\hardware\tools\avr\utils\bin;
CD %TEMP%
MD %PUBLIC%\HEXtemp
for /R %TEMP% %%f in (*.hex) do XCOPY /D /Y %%f %PUBLIC%\HEXtemp\
DIR %PUBLIC%\HEXtemp\*.hex /s /b /O:-D /T:W >HexRhere
SET /P HEX= <HexRhere
ECHO %HEX% >LastHEX.txt
NOTEPAD LastHEX.txt
SET HEX=""

Ray

Some very cool information. I find this very fascinating. I've never worked with assembly language but I know of it and I've seen bits of code online. It doesn't exactly look friendly to use and I imagine if you converted the hex back to assembly it wouldn't be of much use to most. I have no need to convert the hex back to assembly or back to a sketch, I was just wondering how that all worked.

I tried saving my hex file and then went into my sketch and changed just one character and recompiled and did a diff on the hex files. I was expecting hardly any difference and was surprised to see a ton of differences.

I haven't had a chance yet tonight to look at avrdude but that'll be my next fun project. Upload this ascii text file of hex codes and program my arduino :slight_smile:

If that one character pushes everything down one byte, a whole lot of stuff might be different.

Imagine a queue. Each person has a position. One person joins the head of the queue. Now the person who was formerly "first" is now "second", and "second" is now "third" and so on. Quite a ripple effect.

MalcolmV8:
I've never worked with assembly language but I know of it and I've seen bits of code online. It doesn't exactly look friendly to use and I imagine if you converted the hex back to assembly it wouldn't be of much use to most.

it is very friendly - if you are the one who wrote it and if of course, you have the source code - and if you annotate it well. It is very effective if you are in the habit of thinking like a microprocessor. XD

Deciphering (disassembling) code generated by a compiler is another matter - optimisations do some curious things which are in many cases, non-intuitive. In particular, you miss the meaningful labels which you put in the original source. A good disassembler allows you to define such labels and (optimally, dynamically) update the new "source" as well as identifying arrays, strings and such. You work from what would be the output listing from an assembler (which is the final stage of a compiler) showing the code in both hex byte and corresponding ASCII character representation.

For all practical purposes, you can't get the code back. If you simply want a carbon copy, just upload the hex into another chip. If you want to get back the functionality, but make changes, I suggest it would be somewhat faster to simply recode.