Flash arduino over serial via ESP8266 WiFi - FIXED

Hi I am trying to figure out how to flash the hex file from a esp to an arduino.

Basically all I need to have confirmed is that the tx0 and rx0 are linked directly to the uart on the Arduino which I'm 99.9% sure they are. So all I need to do it pull the reset high on the arduino and send the hex file over serial from the esp to the rx pin on the Arduino then once the hex stream has ended pull reset low ? And the flash is done, basically the same process that is used to flash a arduino I don't see why it would not work any thoughts ?

Why on Earth would you flash ESP8266 firmware to an Atmega chip?

Ha sorry, the arduinos hex file being the compiled code is stored on a server the esp gets the files and steams it over serial to the Arduino. I have that working fine. But at the moment I’m just printing the hex file to the serial monitor. Instead I will have the line going to rx0 and a digital pin to the reset.

the dfu lib

arduino version created from arduino-debug branch

used in WiFi Link firmware

some desription of usage

an issue

Thankyou Juraj i just had a look at the library, do you know any good tutorials that use it ? i.e. the connections and usage ?

I edited the post. It is everything what is.

if you go deeper in this, it would be good to make an avrdude network upload compatible esp wifiserver on port 328

There's a library available for programming AVR chips over TCP. It's included with the ESP8266 Arduino core.

Pieter

PieterP:
There’s a library available for programming AVR chips over TCP. It’s included with the ESP8266 Arduino core.

Arduino/Arduino_Wifi_AVRISP.ino at master · esp8266/Arduino · GitHub

Pieter

not over Serial

Is there any reason you can't use the ICSP pins?

MorganS:
Is there any reason you can't use the ICSP pins?

connected to what on an esp module?

Juraj:
connected to what on an esp module?

HSPI

There's no need to reinvent the wheel here (unless it's for educational purposes, of course).

Pieter

for now spi connection of esp to arduino is unexplored for me. just today I searched for esp8266 modules with HSPI pins. no small breakout boards. only plain 12F module has it or big boards like uno sized Wemos D1

Juraj, I found your library and I need your help if possible all I have created is a code for my ESP-12f that can connect to my server and print the entire HEX file to the serial but that's as far as i have got, I have emailed you as I found your email on github, are you up for making a sketch for my ESP8266-12F to flash my arduino Mega, plus the wiring i.e. what pins go to what , making the library and all. cheers from kawasaki

thank you for trusting me with this. I help you, but not as freelancer for money. Only here on forum. Can you post here your code?

What esp module you have exactly? A photo would be best.

How did you connect esp to Mega? In esp sketch I see you use sw serial to keep the Serial free for upload and debugging.

To flash the Atmega from esp over serial, the esp must be connected to RX/TX of Mega (Serial0). Then USB on Mega can't be used. Perhaps the SPI connection suggested by Pieter is a better option.

Wow ty Juraj, Ok i have recorded the code in action from the ESP to a Arduino Uno as i am playing around at the moment and don’t want to brick my Mega2560.

I have a ESP8266-12 using the 12f chip the actual board i am using is a ESP-202 industrial board. I have the Arduino send over some variable to the ESP via software serial on pins 2,3 on the UNO to pins 13,15 on the ESP8266, the values that are sent from the Arduino are received by the ESP8266 and sent to a php script to be added to my database, (this all works perfectly).

The Arduino changes the sysID value “one of the values i sent to the esp” to “YES” and the ESP reads that and then instead connects to a url which hosts the HEX file to flash “Blink.c” , the esp then resets the Arduino and sends the HEX code over serial to the Arduinos RX,TX pins (Serial0) but it does not flash the Arduino. the reset is done via gpio 5 on the ESP to the rst pin on the arduino and the led blinks indicating it has reset.

Soooo, i have missed something in the flash process do i just need to reset the arduino and send the hex file over serial at 115200 to the rx pin ? or am i missing something like a checksum or confirmation response ?

Please look at my code below, the code is simple as can be for now just as proof of concept.

the master is the one that sends the system values i just made a demo code to emulate the variables for now.

and the slave runs on the esp8266-12 and receives the variables, creates a hotspot to enter lan details, and connect to a server to enter the data into a database and connects to a url to get a hex file when the YES cmd is received.

Oh and my code to reset the arduino, i found that until i run the reset low, high loop the arduino keeps resetting, i have the gpio 5 set to an output and set to high on boot yet that makes the arduino constantly reset until the actual gpio 5 low high loop runs just prior to send the hex file.

Watch the video for more details and my code below:

Sorry the code made my post over 9000 characters so i had had to attach the files below.

Video explanation

Im terrible at videos have mercy :o

basicSerialMaster.ino (3.25 KB)

basicSerialSlave_1.1.ino (5.31 KB)

I do not understand the process of flashing the Atmega. I just used the dfu library with WiFi Link as it was integrated by Arduino.org and Davide the author of the dfu library. it is not so easy like sending something after reset. use the library.

esp-link author has some notes on how he implemented the flashing

How does this ESP-Link library work ? do i host the hex file on a server and then run commands from a pc in the shell. and lastly is it just like arduino code in the sense that i can integrate into my code where i print the hex file.

I have google ESP-link for Arduino and i have found the library on github but it has no examples and i cant find a video explaining the code usage or whatever.

From what you say you have used ESP-link before with success, can you show me how to get it working, maybe you have a sketch already ? the process of setting up ESP-link and generating the files and such looks quite confusing well very confusing i have no idea where to start, nor can i see where to host the HEX file and such.

esp-link is a firmware. it takes all esp. it is not an Arduino sketch. it is build with esp sdk tool chain. it is compatible with avrdude network upload, which is executed in IDE when network port is selected for Upload.

look at the dfu library. here is the arduino version. unzip it into your arduino libraries folder.

as example how to use dfu, see ArduinoMcuOTA,ino

default implementation of dfu hooks into WebServer in dfu_binary_file_start_rx. so the upload from PC is HTTP POST/ the tool for it is arduino-tool-mcu-ota.

Ok i have been examining the serial commands the library’s you linked uses too get the Arduino ready for flashing over serial i.e. the SDK500 protocols esp-link serial flash cmds , and i have been looking at the most popular one in python by that spanish dude, link here ESP flash arduino over wifi and i have made a sketch to send the cmds from one arduino to another over the serial0 rx,tx which just for proof of concept the receiving arduino is running a blank sketch.

And omg i got it working, i get all the expected commands back and its ready to receive the hex file, i am yet to try sending the hex file as i need to get the timing just right but i think i just maybe might get it working. Looking at the python code and how he sends the SKD500 commands over is very basic and easy to follow and his works fine, so i really don’t see why mine wont. I pray i get this working soon it would be so cool to just click on button on my arduinos display and the esp connects to the server and flashes the arduino, no silly command prompt or file generation or ports.

Anywho here’s the code i created to get an Arduino into a flash ready state, you need two Arduino’s and the pin connections are just Software serial on 2,3 to TX RX and a ground, 5v to vin and pin 5 to reset.

Any refinement on the code would be appreciated, thank you.

It’s such a shame wifi flashing an Arduino is not a widely done thing Arduino code wise, we need to make this a proper project and turn it into a library if i pull this off as it runs on a esp as Arduino code and allows other codes to run.

Im close very close but this last part may be the hardest, i already can retrieve and print the hex file and i can get the Arduino into a flash state now it just timing the server connection for the hex file as i can’t store it being between 80 Kb,s for Blink and up to 200 Kb,s for my Mega project so it has to be printed out on the fly like you see in my post with the video on the other page.

//Receiver Code
#include <SoftwareSerial.h>
SoftwareSerial Serial1(2, 3); // RX, TX nodemcu

String test;
String test2;
String test3;
String test4;
boolean lock1 = false;
boolean lock2 = false;
boolean lock3 = false;
boolean lock4 = false;

void setup() {
  //delay(5000);
  Serial.begin(115200);
  Serial1.begin(115200);
  pinMode(5, OUTPUT); // gpio which is attaced to Arduino reset pin.
  delay(100);
  digitalWrite(5, LOW); // sends a brief reset pulse
  delay(50); // 50ms works better
  digitalWrite(5, HIGH); // clears reset
  delay(50); // ~50ms later
  Serial1.write("\x20"); // sends CRC_EOP+CRC_EOP
  delay(70); // ~70ms later
}

//esp-link sends a brief reset pulse (1ms)
//esp-link sends CRC_EOP+CRC_EOP ~50ms later
//esp-link sends CRC_EOP+CRC_EOP every ~70-80ms
//eventually optiboot responds with STK_INSYNC+STK_OK (0x14;0x10)
//esp-link sends one CRC_EOP to sort out the even/odd issue
//either optiboot responds with STK_INSYNC+STK_OK or nothing happens for 70-80ms, in which case esp-link sends another CRC_EOP
//esp-link sends STK_GET_SYNC+CRC_EOP and optiboot responds with STK_INSYNC+STK_OK and we're in sync now ------------------------------ TO DO
//esp-link sends the next command (starts with 'u') and programming starts...

void loop() {
  if (!lock1) {
    while (Serial1.available()) {
      test += String(Serial1.read()); // check for responce with STK_INSYNC+STK_OK (0x14;0x10)
    }
    if (test == "2016") { // eventually optiboot responds with STK_INSYNC+STK_OK (0x14;0x10)
      lock1 = true;
    }
    if (!lock1) {
      Serial1.write("\x30\x20");
      delay(70);
    }
  }
  else {
    if (!lock2) {
      while (Serial1.available()) {
        test2 += String(Serial1.read());
      }
      if (test2 == "2016") {
        lock2 = true;
      }
      if (!lock2) {
        Serial1.write("\x20"); // send one CRC_EOP to sort out the even/odd issue
        delay(70);
      }
    }
    else {
      if (!lock3) {
        while (Serial1.available()) {
          test3 += String(Serial1.read());
        }
        if (test3 == "2016") {
          lock3 = true;
        }
        if (!lock3) {
          Serial1.write("\x50\x20");
          delay(70);
        }
      }
      else {
        if (!lock4) {
          while (Serial1.available()) {
            test4 += String(Serial1.read());
            if (test4 == "20") {
              lock4 = true;
              Serial.println("You f***ing legend Flash time ahhhhhhhhhhh!");
            }
          }
          if (!lock4) {
            Serial1.write("\x75\x20");
            delay(70);
          }
        }
        else {
          // print hex here the connection will have to be timed so it happens within 5 secs timeout. 
          // then reset arduino
        }
      }
    }
  }
}

Kapla - Success

wow it worked pic here!

Now i just need to take the code on the Arduino and convert it to run on a ESP alone using a Arduino ESP TCP/IP library, as at the moment the code im am using to flash it runs on the Arduino to be flashed and sends AT commands to a ESP8266-01.

So does anyone know a good TCP/IP Arduino ESP library ?

here’s the code i need to run on th ESp instead of on the Arduino

// Copyright (C) 2014 SISTEMAS O.R.P.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program.  If not, see <http://www.gnu.org/licenses/>.

# define  SEARCH_AFTER_RESET "ready"
# define  INTRO "\r\n"

# define  AP_NORMAL  "EdsWiFi2G"
# define  PWD_NORMAL  "Password90"
# define  HOST_NORMAL  "192.168.0.15"
# define  PORT_NORMAL  "49000"

# define  AP_BOOTLOADER  "EdsWiFi2G"
# define  PWD_BOOTLOADER  "Password90"
# define  HOST_BOOTLOADER  "192.168.0.15"
# define  PORT_BOOTLOADER  "50000"

boolean ok = false;
int counter = 0;

void setup ()
{
  //delay(5000);
  pinMode (13, OUTPUT);
  Serial.begin(115200); // AT+UART_DEF=9600,8,1,0,0
  //Serial.println("Init WI07C...");
  // Remove any garbage from the RX buffer
  //delay (3000);
 // while (Serial.available() > 0) Serial.read();
  ok = init_commands();
}

//-----------------------------------------------------------------------------------------------------------------------------------

void loop()
{
  if (ok)
  {
    digitalWrite(13, HIGH);
    ok = test();
    if (ok && look_for("reboot", 5000))
    {
      if (boot_commands())
      {
        //Serial.println("Starting reboot to flash hex !");
        pinMode (12, OUTPUT);
        digitalWrite (12, LOW);
        for (;;);
      }
      else
      {
        ok = false;
      }
    }
  }
  else
  {
    digitalWrite (13, LOW);
    Serial.println("Error sending AT commands");
    for (;;);
  }
}

//-----------------------------------------------------------------------------------------------------------------------------------

/*
  Parameters:
  command: The string to send.
  timeout: The maximum time in milliseconds the function can be running before a time out.
  wait_for1: The search string when the command succeeded.
  wait_for1: The search string when the command failed.
  Returns:
  The string contained in wait_for1, wait_for2 or the string TIMEOUT.
  Description:
  It sends the command trough the serial port and waits for one of the expected strings or exit with timeout
*/
String send (String command, int timeout, String wait_for1, String wait_for2)
{
  unsigned long time = millis();
  String received = "";
  Serial.print(command);
  Serial.print(INTRO);

  while (millis() < (time + timeout))
  {
    if (Serial.available() > 0)
    {
      received.concat (char(Serial.read()));
      if (received.indexOf(wait_for1) > - 1)
      {
        return wait_for1;
      }
      else if (received.indexOf(wait_for2) > - 1)
      {
        return wait_for2;
      }
    }
  }
  return "TIMEOUT";
}
//-----------------------------------------------------------------------------------------------------------------------------------
/*
  Parameters:
  wait_for: The search string.
  timeout: The maximum time in milliseconds the function can be running before a time out.
  Returns:
  True if the string was found, otherwise False.
  Description:
  It waits for the expected string.
*/
boolean look_for (String wait_for, int timeout)
{
  unsigned long time = millis();
  String received = "";
  while (millis() < (time + timeout))
  {
    if (Serial.available() > 0)
    {
      received.concat (Serial.readString());
      if (received.indexOf(wait_for) > - 1)
      {
        return true;
      }
    }
  }
  return false;
}
//-----------------------------------------------------------------------------------------------------------------------------------
/*
  Parameters:
  command: The string to send.
  timeout: The maximum time in milliseconds the function can be running before a timeout.
  wait_for1: The search string when the command succeeded.
  wait_for1: The search string when the command failed.
  Returns:
  True if the wait_for1 string was found, otherwise False.
  Description:
  It sends the command trough the serial port and waits for one of the expected strings or exit with timeout.
*/
boolean send_command (String command, int timeout, String wait_for1, String wait_for2)
{
  String state;
  state = send (command, timeout, wait_for1, wait_for2);
  if (state == wait_for1)
  {
    return true;
  }
  else if (state == wait_for2)
  {
    // do something on error
  }
  else
  {
    // do something on timeout
  }
  return false;
}
//-----------------------------------------------------------------------------------------------------------------------------------
/*
  Parameters:
  Nothing
  Returns:
  True if all commands were successfully, otherwise False.
  Description:
  It initializes the module, joins to the access point and connects to the server.
*/
boolean init_commands()
{
  // Serial_display ("Changing mode");
  if (send_command ("AT+CWMODE=1", 5000, "OK", "ERROR"))
  {
    //Serial_display ("Resetting module");
    Serial.println("AT+RST\r\n");
    delay(5000);
    //   {
    //Serial_display ("Joining AP") ;
    String cwjap = "AT+CWJAP=\"";
    cwjap += AP_NORMAL;
    cwjap += "\",\"";
    cwjap += PWD_NORMAL;
    cwjap += "\"" ;
    Serial.println(cwjap + "\r\n");
    delay(5000);
    Serial.println("AT+CIPMUX=0\r\n");
    delay(2000);
    Serial.println("AT+CIPMODE=0\r\n");
    delay(2000);
    Serial.println("AT+CIPSTART=\"TCP\",\"192.168.0.15\",4900\r\n");
    delay(10000);
    return true;
  }
  //return false;
}
//-----------------------------------------------------------------------------------------------------------------------------------
/*
  Parameters:
  Nothing
  Returns:
  True if all commands were successfully, otherwise False.
  Description:
  It initializes the module, joins to the access point, connects to the server and starts the protocol.
*/
boolean boot_commands () 
 { 
   //Serial.println("AT+RST\r\n");
   delay(5000);  
   String cwjap = "AT+CWJAP=\""; 
   cwjap += AP_BOOTLOADER; 
   cwjap += "\",\""; 
   cwjap += PWD_BOOTLOADER; 
   cwjap += "\""; 
   if (send_command (cwjap, 20000, "OK", "FAIL")) 
     if (send_command ("AT+CIPMUX=0", 2000, "OK", "ERROR")) 
       if (send_command ("AT+CIPMODE=1", 2000, "OK", "ERROR")) 
       {        
         String cipstart = "AT+CIPSTART=\"TCP\",\""; 
         cipstart += HOST_BOOTLOADER; 
         cipstart += "\","; 
         cipstart += PORT_BOOTLOADER; 
         if (send_command(cipstart, 5000, "OK", "ERROR")) 
           if (send_command("AT+CIPSEND", 2000, ">", "ERROR")) 
           { 
             if (send_command("hello", 2000, "welcome", "error"))  // increase from 2000 to 500
               if (send_command("Blink.ino.hex", 5000, "ok", "error")) 
                 return true; 
           } 
       }            
   return false; 
 } 
//-----------------------------------------------------------------------------------------------------------------------------------
/*
  Parameters:
  Nothing
  Returns:
  True if all commands were successfully, otherwise False.
  Description:

  It sends a string to the remote host and show it in the display.
*/
boolean test ()
{
  String command = "AT+CIPSEND=";
  String to_send = "hello guys!";
  to_send += counter;
  command += to_send.length() + 2;
  if (send_command( command, 5000, ">", "ERROR"))
    if (send_command(to_send + "\r\n", 5000, "OK", "ERROR"))
    {
      Serial.println(to_send);
      counter ++;
      return true;
    }
  return false;
}