Download an updated sketch remotely

FrankLevis · February 20, 2020, 10:03am

Good morning,
to update my arduino I am currently using Jurai's excellent ArduinoOTA libraries. But what I would like to do is another thing, but I am not able to find a solution for this board.

I would like the arduino to check every day if there is an updated version of the sketch on a server. If it exists it should download it to the SD and then install it. Has anyone done it already or could you give me some indication? Thanks a lot and sorry for my bad english.

Juraj · February 20, 2020, 10:27am

there is a new example

github.com

jandrassy/ArduinoOTA/blob/master/examples/Advanced/OTASketchDownload/OTASketchDownload.ino

/*

 This example downloads sketch update over network.
 It doesn't start the OTA upload sever of the ArduinoOTA library,
 it only uses the InternalStorage object of the library
 to store and apply the downloaded binary file.

 To create the bin file for update of a SAMD board (except of M0),
 use in Arduino IDE command "Export compiled binary".
 To create a bin file for AVR boards see the instructions in README.MD.
 To try this example, you should have a web server where you put
 the binary update.
 Modify the constants below to match your configuration.

 Created for ArduinoOTA library in February 2020
 by Juraj Andrassy
 */

#include <ArduinoOTA.h> // only for InternalStorage
#include <Ethernet.h>

This file has been truncated. show original

the SD version will follow

EDIT: the SD version

github.com

jandrassy/ArduinoOTA/blob/master/examples/Advanced/OTASketchDownload_SD/OTASketchDownload_SD.ino

/*

 This example downloads sketch update over network to SD card.
 It doesn't use the ArduionOTA library at all. It is intended for use
 with the SDU library of SAMD boards package or with SD bootloader for AVR.

 To create the bin file for update of a SAMD board (except of M0),
 use in Arduino IDE command "Export compiled binary".
 To create a bin file for AVR boards see the instructions in README.MD.
 To try this example, you should have a web server where you put
 the binary update.
 Modify the constants below to match your configuration.

 Created for ArduinoOTA library in February 2020
 by Juraj Andrassy
 */

#include <Ethernet.h>
#include <ArduinoHttpClient.h>
#include <SD.h>

This file has been truncated. show original

FrankLevis · February 20, 2020, 11:28am

Check it now!
Thanks so much for your help and work.

FrankLevis · February 20, 2020, 11:47am

I see that it uses the ethernet library. I have to use the wifinina. Do you think it is possible to adapt it? I try.

Juraj · February 20, 2020, 12:43pm

FrankLevis:
I see that it uses the ethernet library. I have to use the wifinina. Do you think it is possible to adapt it? I try.

you know how to connect to network?
so only replace EthernetClient with WiFiClient in handleSketchDownload()

FrankLevis · February 20, 2020, 2:49pm

Okay, I tried a few rate change. Keep in mind that I'm not a programmer so I'm having some difficulty.

The first problem is the call to the HttpClient.h library
So another library calls me and gives me errors.
I changed to ArduinoHttpClient.h and this part is solved.

Then I used the wifinina library instead of Ethernet and applied the changes to the code which I'm not sure about but everything seems to be fine.

Compile, no mistake.
Loading on arduino, no errors.

Arduino connects via wifi, then to the web server. Find the file, check it and start the download. Here hangs in half for time out.

Check for update file /update-v2.bin
Update status code: 200
Server returned update file of size 42484 bytes
Timeout downloading update file at 6865 bytes. Can't continue with update.

the code:

/*
  This example downloads sketch update over network.
  It doesn't start the OTA upload sever of the ArduinoOTA library,
  it only uses the InternalStorage object of the library
  to store and apply the downloaded binary file.
  To create the bin file for update, use in Arduino IDE command
  "Export compiled binary".
  To try this example, you should have a web server where you put
  the binary update.
  Modify the constants below to match your configuration.
  Created for ArduinoOTA library in February 2020
  by Juraj Andrassy
*/

#include <WiFiNINA.h>
#include <ArduinoOTA.h>
#include <ArduinoHttpClient.h> // changed from HttpClient.h
#include "arduino_secrets.h"

char ssid[] = SECRET_SSID;
char pass[] = SECRET_PASS;
int status = WL_IDLE_STATUS;

const short VERSION = 1;
#define Serial SerialUSB
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

void handleSketchDownload() {

  const char* SERVER = "192.168.103.179"; // must be string for HttpClient
  const unsigned short SERVER_PORT = 8080;
  const char* PATH = "/update-v%d.bin";
  const unsigned long CHECK_INTERVAL = 5000;

  static unsigned long previousMillis;

  unsigned long currentMillis = millis();

  if (currentMillis - previousMillis < CHECK_INTERVAL)
    return;
  previousMillis = currentMillis;



  // EthernetClient transport;
  WiFiClient transport;
  HttpClient client(transport, SERVER, SERVER_PORT);

  char buff[32];
  snprintf(buff, sizeof(buff), PATH, VERSION + 1);

  Serial.print("Check for update file ");
  Serial.println(buff);

  client.get(buff);

  int statusCode = client.responseStatusCode();
  Serial.print("Update status code: ");
  Serial.println(statusCode);
  if (statusCode != 200) {
    client.stop();
    return;
  }

  int length = client.contentLength();
  if (length == HttpClient::kNoContentLengthHeader) {
    client.stop();
    Serial.println("Server didn't provide Content-length header. Can't continue with update.");
    return;
  }
  Serial.print("Server returned update file of size ");
  Serial.print(length);
  Serial.println(" bytes");

  if (!InternalStorage.open(length)) {
    client.stop();
    Serial.println("There is not enough space to store the update. Can't continue with update.");
    return;
  }
  byte b;
  while (length > 0) {
    if (!client.readBytes(&b, 1)) // reading a byte with timeout
      break;
    InternalStorage.write(b);
    length--;
  }
  InternalStorage.close();
  client.stop();
  if (length > 0) {
    Serial.print("Timeout downloading update file at ");
    Serial.print(length);
    Serial.println(" bytes. Can't continue with update.");
    return;
  }

  Serial.println("Sketch update apply and reset.");
  Serial.flush();
  InternalStorage.apply(); // this doesn't return
}

void setup() {
  //Initialize serial and wait for port to open:
  Serial.begin(115200);
  while (!Serial) {
    ;
  }

  Serial.print("Sketch version ");
  Serial.println(VERSION);

  if (WiFi.status() == WL_NO_MODULE) {
    Serial.println("Communication with WiFi module failed!");
    while (true);
  }

  String fv = WiFi.firmwareVersion();
  if (fv < WIFI_FIRMWARE_LATEST_VERSION) {
    Serial.println("Please upgrade the firmware");
  }

  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network:
    status = WiFi.begin(ssid, pass);

    // wait 10 seconds for connection:
    delay(10000);
  }

  // you're connected now, so print out the data:
  Serial.print("You're connected to the network");
  printCurrentNet();
  printWifiData();

}

void loop() {
  // check for updates
  handleSketchDownload();

  // add your normal loop code below ...
}


void printWifiData() {
  // Stampa del indirizzo IP
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // Stampa del MAC Address
  byte mac[6];
  WiFi.macAddress(mac);
  Serial.print("MAC address: ");
  Serial.println(bytesToMACAddress(mac));
}

// Stampa dei dati di connessione: SSID, BSSID, RSSI e Criptazione
void printCurrentNet() {
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  byte bssid[6];
  WiFi.BSSID(bssid);
  Serial.print("BSSID: ");
  Serial.println(bytesToMACAddress(bssid));

  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI):");
  Serial.println(rssi);

  byte encryption = WiFi.encryptionType();
  Serial.print("Encryption Type:");
  Serial.println(encryption, HEX);
  Serial.println();
}

// Costruzione del MAC Address
String getMACAddress()  {
  byte mac[6];
  WiFi.macAddress(mac);

  return bytesToMACAddress(mac);
}

String bytesToMACAddress(byte mac[]) {
  String m = "";

  for (int i = 5; i >= 0; i--) {
    m = m + byeToHex(mac[i]);
    if (i > 0) {
      m = m + ":";
    }
  }

  return m;
}

String byeToHex(byte b) {
  if (b < 16) {
    return "0" + String(b, HEX);
  }

  return String(b, HEX);
}

Juraj · February 20, 2020, 3:15pm

the timeout is 1 second. you can try to set 2 seconds with
client.setTimeout(2000);
before while (length > 0) {

FrankLevis · February 20, 2020, 3:27pm

I tried, selecting first two seconds, then 4 and finally 8. It seems that nothing changes.

Check for update file /update-v2.bin
Update status code: 200
Server returned update file of size 42484 bytes
Timeout downloading update file at 34149 bytes. Can't continue with update.
Check for update file /update-v2.bin
Update status code: 200
Server returned update file of size 42484 bytes
Timeout downloading update file at 26969 bytes. Can't continue with update.
Check for update file /update-v2.bin
Update status code: 200
Server returned update file of size 42484 bytes
Timeout downloading update file at 2557 bytes. Can't continue with update.
Check for update file /update-v2.bin

FrankLevis · February 20, 2020, 3:48pm

I set a 15 second timeout and now it works!
tomorrow other tests and than I optimize

thanks!

Juraj · February 20, 2020, 6:20pm

FrankLevis:
I set a 15 second timeout and now it works!
tomorrow other tests and than I optimize

thanks!

15000 miliseconds timeout?

FrankLevis · February 21, 2020, 7:15am

client.setTimeout(15000);

FrankLevis · February 21, 2020, 7:34am

This is the complete and working sketch with a MKR1010 in wifi and the MKR env shield module.
Maybe it's not perfect but it works If it can be useful it is at your disposal.

Thanks so much for all your help

/*
  This example downloads sketch update over network.
  It doesn't start the OTA upload sever of the ArduinoOTA library,
  it only uses the InternalStorage object of the library
  to store and apply the downloaded binary file.
  To create the bin file for update, use in Arduino IDE command
  "Export compiled binary".
  To try this example, you should have a web server where you put
  the binary update.
  Modify the constants below to match your configuration.
  Created for ArduinoOTA library in February 2020
  by Juraj Andrassy
*/

#include <WiFiNINA.h>
#include <ArduinoOTA.h>
#include <ArduinoHttpClient.h> // changed from HttpClient.h
#include "arduino_secrets.h"

char ssid[] = SECRET_SSID;
char pass[] = SECRET_PASS;
int status = WL_IDLE_STATUS;

const short VERSION = 3;
#define Serial SerialUSB
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

void handleSketchDownload() {

  const char* SERVER = "192.168.103.179"; // must be string for HttpClient
  const unsigned short SERVER_PORT = 8080;
  const char* PATH = "/update-v%d.bin";
  const unsigned long CHECK_INTERVAL = 5000;

  static unsigned long previousMillis;

  unsigned long currentMillis = millis();

  if (currentMillis - previousMillis < CHECK_INTERVAL)
    return;
  previousMillis = currentMillis;

  // EthernetClient transport;
  WiFiClient transport;
  HttpClient client(transport, SERVER, SERVER_PORT);

  char buff[32];
  snprintf(buff, sizeof(buff), PATH, VERSION + 1);

  Serial.print("Check for update file ");
  Serial.println(buff);

  client.get(buff);

  int statusCode = client.responseStatusCode();
  Serial.print("Update status code: ");
  Serial.println(statusCode);
  if (statusCode != 200) {
    client.stop();
    return;
  }

  int length = client.contentLength();
  if (length == HttpClient::kNoContentLengthHeader) {
    client.stop();
    Serial.println("Server didn't provide Content-length header. Can't continue with update.");
    return;
  }
  Serial.print("Server returned update file of size ");
  Serial.print(length);
  Serial.println(" bytes");

  if (!InternalStorage.open(length)) {
    client.stop();
    Serial.println("There is not enough space to store the update. Can't continue with update.");
    return;
  }
  byte b;

  client.setTimeout(30000); //timeout per download
  
  while (length > 0) {
    if (!client.readBytes(&b, 1)) // reading a byte with timeout
      break;
    InternalStorage.write(b);
    length--;
  }
  InternalStorage.close();
  client.stop();

  if (length > 0) {
    Serial.print("Timeout downloading update file at ");
    Serial.print(length);
    Serial.println(" bytes. Can't continue with update.");
    return;
  }

  Serial.println("Sketch update apply and reset.");
  Serial.flush();
  InternalStorage.apply(); // this doesn't return
}

void setup() {
  //Initialize serial and wait for port to open:
  Serial.begin(115200);
  while (!Serial) {
    ;
  }
  Serial.print("Sketch version ");
  Serial.println(VERSION);

  if (WiFi.status() == WL_NO_MODULE) {
    Serial.println("Communication with WiFi module failed!");
    while (true);
  }

  String fv = WiFi.firmwareVersion();
  if (fv < WIFI_FIRMWARE_LATEST_VERSION) {
    Serial.println("Please upgrade the firmware");
  }

  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    status = WiFi.begin(ssid, pass);
    delay(10000);
  }
  Serial.print("You're connected to the network");
  printCurrentNet();
  printWifiData();
}

void loop() {
  // check for updates
  handleSketchDownload();

  // add your normal loop code below ...
}


void printWifiData() {
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);
  byte mac[6];
  WiFi.macAddress(mac);
  Serial.print("MAC address: ");
  Serial.println(bytesToMACAddress(mac));
}

void printCurrentNet() {
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  byte bssid[6];
  WiFi.BSSID(bssid);
  Serial.print("BSSID: ");
  Serial.println(bytesToMACAddress(bssid));

  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI):");
  Serial.println(rssi);

  byte encryption = WiFi.encryptionType();
  Serial.print("Encryption Type:");
  Serial.println(encryption, HEX);
  Serial.println();
}

String getMACAddress()  {
  byte mac[6];
  WiFi.macAddress(mac);

  return bytesToMACAddress(mac);
}

String bytesToMACAddress(byte mac[]) {
  String m = "";

  for (int i = 5; i >= 0; i--) {
    m = m + byeToHex(mac[i]);
    if (i > 0) {
      m = m + ":";
    }
  }

  return m;
}

String byeToHex(byte b) {
  if (b < 16) {
    return "0" + String(b, HEX);
  }

  return String(b, HEX);
}

Juraj · February 21, 2020, 7:47am

the SD version

github.com

jandrassy/ArduinoOTA/blob/master/examples/Advanced/OTASketchDownload_SD/OTASketchDownload_SD.ino

/*

 This example downloads sketch update over network to SD card.
 It doesn't use the ArduionOTA library at all. It is intended for use
 with the SDU library of SAMD boards package or with SD bootloader for AVR.

 To create the bin file for update of a SAMD board (except of M0),
 use in Arduino IDE command "Export compiled binary".
 To create a bin file for AVR boards see the instructions in README.MD.
 To try this example, you should have a web server where you put
 the binary update.
 Modify the constants below to match your configuration.

 Created for ArduinoOTA library in February 2020
 by Juraj Andrassy
 */

#include <Ethernet.h>
#include <ArduinoHttpClient.h>
#include <SD.h>

This file has been truncated. show original

FrankLevis · February 21, 2020, 8:04am

Excellent news! I look at it immediately!

Juraj:
the SD version
ArduinoOTA/examples/Advanced/OTASketchDownload_SD/OTASketchDownload_SD.ino at master · JAndrassy/ArduinoOTA · GitHub

Juraj · February 23, 2020, 5:42pm

I tested with WiFiNina. no problems. it worked at first attempt

ninora · March 25, 2020, 4:25pm

Juraj,
This is fantastic! much needed utility.
I did see a timeout so added client.setTimeout(5000);, other than that, fabulous!
Thanks a lot!

ninora · March 25, 2020, 7:52pm

To avoid restarting on bad binary I am looking for a way to verify the downloaded file checksum using CRC32 (or other method?).

Is it possible to access the written buffer once the read/write loop completes so I don't have to hold another buffer in memory?

Juraj · March 25, 2020, 9:05pm

TCP verifies the transfer over network. The library verifies the size. How do you want to send the CRC?
if you use the SD version, you can check the file.

ninora · March 25, 2020, 9:21pm

In my implementation the board receives an instruction from the server to update itself with server/port/path information, with that I can send the expected CRC.
I am not using SD, just the boards (MKR and Nano 33).

Juraj · March 26, 2020, 6:22am

ninora:
In my implementation the board receives an instruction from the server to update itself with server/port/path information, with that I can send the expected CRC.
I am not using SD, just the boards (MKR and Nano 33).

if you your code is based on OTASketchDownload, then you can add the calculation here

 while (length > 0) {
   if (!client.readBytes(&b, 1)) // reading a byte with timeout
     break;
   InternalStorage.write(b);
   length--;
 }

b is one byte copied from download stream into internal storage

Topic		Replies	Views
Uploading sketches Over-the-Air (OTA) on an Arduino MKR NB 1500 General Guidance	4	484	September 27, 2023
Remotely Updating Arduino mega Sketch using enc28j60 ethernet Module 3rd Party Boards	8	658	July 11, 2023
ArduinoOTA GET with SAMD Programming	4	1540	May 6, 2021
ArduinoOTA GET with SAMD Jobs and Paid Consultancy	3	1342	May 6, 2021
Arduino Mega2560 OTA Français	9	454	January 2, 2023

Download an updated sketch remotely

Related topics