Programming ESP8266 Without Compiling EVERY Time

Hi guys,

Simple question: every time I upload to my ESP the Arduino IDE compiles it, which is really annoying because it takes a while, especially for the first time I open the IDE. Is there a way to somehow get the .hex file from the Arduino IDE (I know where to get this) and flash it to the ESP using a different program without compiling every time? I'm assuming what I want to do is not possible with Arduino IDE.

Note: I know how to upload hex files to AVR chips, but this ain't AVR so that's why I'm asking.

If you have any other way of doing this I'm willing to hear it!

Thanks!

Do this:

  • File > Preferences > Show verbose output during: > upload (check) > OK
  • Sketch > Upload

After the upload completes examine the output in the black console window at the bottom of the Arduino IDE window. You will find the command the Arduino IDE used to upload to the ESP8266, which you can run from the command line. You may need to scroll the console window up to find it because the output is longer than will fit in the small window.

Best advice I can give you: use Linux.
Compilation times are 2 to 3 times faster:

Ubuntu 16.04 Windows 10
Initial compilation: 11.5s 23.8s
Normal compilation: 1.8s 5.9s

(Tested on an i7-7700HQ with the FSBrowser example.)

Pieter

Thanks for the replies! I was wondering about that line of info too, much like the command line for avrdude. However, I brushed that off as something AVR-specific. Here's the line that I'm getting from verbose output:

C:\Users\{My Name}\AppData\Local\Arduino15\packages\esp8266\tools\esptool\0.4.9/esptool.exe -vv -cd nodemcu -cb 921600 -cp COM5 -ca 0x00000 -cf C:\Users\{User Name}\AppData\Local\Temp\arduino_build_534439/Sketch_Name.ino.bin

So how exactly do I go about flashing that with command line? Do I need to go and find that .bin file and save it somewhere and change the command to reference that saved file?

UPDATE: I got it working!!! Thanks so much!

For those of you wondering:

  • Enable verbose output under Preferences by checking "Upload"

  • Upload to the ESP board with Arduino IDE as usual

  • The verbose output will include something like this:

C:\Users\{YOUR NAME}\AppData\Local\Arduino15\packages\esp8266\tools\esptool\0.4.9/esptool.exe -vv -cd nodemcu -cb 921600 -cp COM5 -ca 0x00000 -cf C:\Users\{YOUR 
 NAME}\AppData\Local\Temp\arduino_build_534439/{SKETCH_NAME}.ino.bin
  • Navigate to the .bin file and copy/paste it to a permanent directory. Basically you need to grab that .bin file from the temp folder and put it in a more permanent place where you can keep it safe.

  • Rename the file by removing the ".ino" portion of the filename, just for book-keeping's sake. Copy the file directory.

  • Open command prompt and run the following command, making sure the baud rate and correct COM port and entered:

"C:\Users\$Users\AppData\Local\Arduino15\packages\esp8266\tools\esptool\0.4.9/esptool.exe" -vv -cd nodemcu -cb 921600 -cp COM5 -ca 0x00000 -cf "C:\Users\$Users\Desktop/SketchName.bin"
  • You should see it load with "............ [24%]" etc.

  • Open Putty for serial debug if needed and you're done!

There is a very simple way to upload faster your sketch to an ESP.

After selecting your board Menu > Tool (e.g. NodeMCU 1.0 (ESP 12E Module)), then select upload speed 921600 instead of 115200.

Yes, I do that already.

Now that I got this working, how would I upload using OTA? I can get OTA to work with Arduino, but how would I do this with only the command line? I read somewhere online that I should use the "espota" tool but the esp8266 Arduino package doesn't include "espota.exe" or "espota.py" so I'm really not sure (but the ESP32 Arduino package does).

Also, it would be great if I could mass-upload to multiple devices at the same time via OTA. Is that even possible?

Any ideas here?

On Ubuntu 16.04:

python /home/$USER/.arduino15/packages/esp8266/hardware/esp8266/2.3.0/tools/espota.py -i 192.168.1.3 -p 8266 --auth=password -f /tmp/arduino_build_######/$FILENAME.ino.bin

Pieter

Awesome, thanks I will try that. I found the espota.py under the hardware folder like you said. I was looking at the wrong folder before.

How do you easily get the IP address of the ESP8266 with OTA because with OTA I'm not going to have it hooked up to serial so I can't just read it from Putty or Arduino serial monitor?

EDIT: Also, I'm getting syntax errors trying to do the command in Windows. I'm not using your exact command of course, but even with what I think should work isn't working. I'm navigating to the folder that espota.py is in, and that command works, but then the command

python espota.py -i 192.168.1.73 -p 8266 --auth=password -f "C:\Users\$User\Desktop/$CodeName.bin"

gives me "invalid syntax". Any ideas?

On linux, you could use:

avahi-browse _arduino._tcp --resolve -p

This outputs:

+;wlp2s0;IPv4;ESP8266;_arduino._tcp;local
=;wlp2s0;IPv4;ESP8266;_arduino._tcp;local;esp8266.local;192.168.1.6;8266;"auth_upload=yes" "board=ESP8266_WEMOS_D1MINI" "ssh_upload=no" "tcp_check=no"

You could then write a simple script that loops over all these IP addresses and starts a thread that runs the python upload script for each ESP8266 it found.

Pieter

Sorry, I'm a very noobish Windows user with no real python experience haha.

avahi-browse just scans for mDNS services. This is how the Arduino IDE discovers OTA network devices:

Your python syntax problems might have to do with the python version that is used. You have to use 2.7, it's possible that the default version in Windows is 3.6, while on Ubuntu, the default version is 2.7.

Pieter

How would I do the avahi-browse on Windows? Again, sorry but I'm a total noob at this sort of stuff. I've only really every used the Arduino IDE environment and I just started using command line for uploading code to multiple devices.

By the way, here's the command that worked for me:

python.exe "C:\Users\$User\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0/tools/espota.py" -i 192.168.1.73 -p 8266 --auth=password123 -f "C:\Users\$User\Desktop/$CodeName.bin"

so basically the script should scan that IP address and run that command after inserting the correct IP address.

mDNS on Windows is quite tricky. You have to install Apple's Bonjour (that comes with iTunes or Safari)*. Then you can use:

dns-sd -B _arduino._tcp

Not sure how to resolve that to an IP address.
(*) Don't take my word for it, I'm not an expert when it comes to mDNS on Windows. There might be a better way.

I'd strongly recommend Linux for this kind of stuff.

If you're familiar with Java, you could check the Arduino source code to see how it's done in the IDE. (See link in my previous reply.)

Pieter

OK, that sounds a bit out of my scope. However, is there an already-made program that I can use that will scan for the IP addresses in the WiFi network? I don't need any super-fancy UI with thousands of numbers everywhere; I just want to see the IP addresses and names.

Also, how hard would it be to create some sort of UI that would allow me to type the IP address and maybe other things like the OTA password or code file directory and after I hit Enter it will basically run the command line for me? But it won't do automatic IP searching, etc.

Thanks!

Arduino uses mDNS for OTA device discovery. Windows doesn't support mDNS directly. So you have to either find a tool that can scan/browse mDNS on the network, or write your own script. At first glance, I can't seem to find any Windows tools that do this, but you could write a Java script, using the JmDNS library. This is how it's done by the Arduino IDE, but there probably are mDNS implementations in other scripting languages as well.

If you just want to know what's going on on your network, you could try an app like "Fing". I don't think it supports mDNS, but it shows all connected devices, their IP addresses and hostnames.

If your modem/router has a DNS server, it might scan the network for WiFi hostnames of connected devices, and include them in its DNS tables.
For example, if the ESP's hostname is "ESP_123456", then it could be added to DNS as "ESP_123456.lan".
If that's the case you could use nslookup instead of dns-sd to get the IP addresses of the ESPs on the network.
You can set the hostname using:

WiFi.hostname("ESP_123456");

The default is ESP_######, where ###### are the last 6 digits of the MAC address.

Alternatively, you could just try a ping sweep, and try all devices (including non-ESP). Maybe you could check for open ports (OTA uses UDP port 8266 IIRC).

Or you could just use Linux, of course :wink:

Pieter

androidfanboy:
Also, how hard would it be to create some sort of UI that would allow me to type the IP address and maybe other things like the OTA password or code file directory and after I hit Enter it will basically run the command line for me? But it won't do automatic IP searching, etc.

Not too hard, if you pick the right scripting language. Python + Tkinter is the first thing that came to mind, but there might be a better solution, depending on what languages you're familiar with.

If you don't need a GUI, you could use a Bash/BAT/PowerShell script.
For instance:

read -p "Path to binary: " path
read -p "Password: " password
read -p "IP address: " ip

python2 /home/$USER/.arduino15/packages/esp8266/hardware/esp8266/2.3.0/tools/espota.py -i $ip -p 8266 --auth="$password" -f "$path"

Pieter