Arduino Zero OTA update over GPRS

We have developed an Arduino Zero based custom device that has also internal SPI flash memory and GPRS modem for connectivity.
What we would like to achieve is a code that downloads a new firmware bin file over a FTP server, save it to the external SPI flash and then perform a firmware update on the internal flash.
That way we can remotely update the firmware on the board.

Can anyone please advise how we can achieve this?

From my understanding:

  1. We should export a bin file with the newer firmware
  2. Download it to the external flash of the custom device using the FTP
  3. Use a custom bootloader to write the external flash addresses to the internal flash and update code?

Is a custom bootloader what we really need? Is there any other way with stock arduino zero bootloader?

Thank you very much!

see the SDU library.

or you can upload the bin to upper half of the internal flash and then move it to running location like the InternalStorage class of the WiFi101OTA library does

Hi Juraj,

thank you very much for your feedback. So from my understanding from the WIFI101 OTA library:

  1. i should export a binary firmware file from the Arduino IDE
  2. Upload it to my FTP server
  3. Download it using GPRS and copy it to the internal flash with a code similar to this
while (client.connected() && read < contentLength) {
      while (client.available()) {
        read++;

        _storage->write((char)client.read());
      }
    }

    _storage->close();

_storage->apply();
  1. Then the micro would restart and run on the new code

I would like to ask the following:

a) Do i need a custom bootloader or i could use the stock Arduino Zero for this to work

b) How should i export the binary file for the OTA? Do i need a special linker script or i just use the binary from the "Sketch / Export Compiled Binary" menu?

Thank you very much !

sigalabs:
I would like to ask the following:

a) Do i need a custom bootloader or i could use the stock Arduino Zero for this to work

b) How should i export the binary file for the OTA? Do i need a special linker script or i just use the binary from the "Sketch / Export Compiled Binary" menu?

a) bootloader is not involved
b) use "Export compiled binary" in Sketch menu. the output is in the sketch folder

Thank you very much Juraj, i managed to make it work using the internal flash method of the WIFI101OTA as you proposed.

The steps i have followed:

  1. Init storage with file size
    WiFiOTA._storage->open(8508); // firmware size as value

  2. Write all bytes through looping
    WiFiOTA._storage->write((char)incoming_data);

  3. Close and apply! Your new firmware is ready!
    WiFiOTA._storage->close();
    WiFiOTA._storage->apply();

sigalabs:
Thank you very much Juraj, i managed to make it work using the internal flash method of the WIFI101OTA as you proposed.

The steps i have followed:

  1. Init storage with file size
    WiFiOTA._storage->open(8508); // firmware size as value

  2. Write all bytes through looping
    WiFiOTA._storage->write((char)incoming_data);

  3. Close and apply! Your new firmware is ready!
    WiFiOTA._storage->close();
    WiFiOTA._storage->apply();

If you included the library, InternalStorage object exists.
open() doesn't use the parameter.

I am thinking about the same.... is there any manuals/howto`s for using WiFI101OTA for updating sketch directly from an HTTP/FTP server? Have been searching for hours and cannot find any. Sigalabs, would be appreciated if you could share in more details / post example sketch for acheiving this.

BR,
Sindre

sindrevr:
I am thinking about the same.... is there any manuals/howto`s for using WiFI101OTA for updating sketch directly from an HTTP/FTP server? Have been searching for hours and cannot find any. Sigalabs, would be appreciated if you could share in more details / post example sketch for acheiving this.

BR,
Sindre

the library is for upload from IDE. but the InternalStorage object of WiFI101OTA library can be 'feed' from any source.

I am trying to update my Arduino Mega OTA.
I burned Optiboot to my board to be able to use ArduinoOTA library, and I am using SIM800L module. The sketch is exported as .hex file, and uploaded to the server.

My idea is to download the .hex file from server and use ArduinoOTA to perform something like @Sigilabs suggested. This is the code which is supposed to do the trick :

char *binaryUpdate;
    inet.httpGET("server", 80, "path to hex", binaryUpdate, 1000);

    ArduinoOTA._storage->open(1000);

    int i = 0;
    while (binaryUpdate[i] != '\0')
    {
        ArduinoOTA._storage->write(binaryUpdate[i]);
        i++;
    }

    ArduinoOTA._storage->close();
    ArduinoOTA._storage->apply();

The problem is that the inetGSM library which I use to download the sketch, stores it to the binaryUpdate variable, and there is not enough memory to do that. What I want instead, is to write the downloaded file directly to ArduinoOTA's InternalStorage, character by character, as they are downloaded from the server.

Can anyone point me to the right direction on how to do that ? GRPS is the only way to perform download, I cannot use WiFi.

vukovic:
I am trying to update my Arduino Mega OTA.
I burned Optiboot to my board to be able to use ArduinoOTA library, and I am using SIM800L module. The sketch is exported as .hex file, and uploaded to the server.

My idea is to download the .hex file from server and use ArduinoOTA to perform something like @Sigilabs suggested. This is the code which is supposed to do the trick :

char *binaryUpdate;

inet.httpGET("server", 80, "path to hex", binaryUpdate, 1000);

ArduinoOTA._storage->open(1000);

int i = 0;
    while (binaryUpdate[i] != '\0')
    {
        ArduinoOTA._storage->write(binaryUpdate[i]);
        i++;
    }

ArduinoOTA._storage->close();
    ArduinoOTA._storage->apply();




The problem is that the inetGSM library which I use to download the sketch, stores it to the binaryUpdate variable, and there is not enough memory to do that. What I want instead, is to write the downloaded file directly to ArduinoOTA's InternalStorage, character by character, as they are downloaded from the server.

Can anyone point me to the right direction on how to do that ? GRPS is the only way to perform download, I cannot use WiFi.

use bin file, because the hex file has not only sketch data. it has lines with addresses a check sum and other control values

I googled how to do it, because I get only .hex generated, and I've found your reply here :

Which states :

For AVR boards "Export compiled binary" generates hex.

If I understand correctly, Arduino Mega 2560 is AVR board ? Can I somehow still generate .bin ?

vukovic:
I googled how to do it, because I get only .hex generated, and I've found your reply here :

Why is there no hex file generated when I "build binary output" from the Arduino IDE? - Arduino Stack Exchange

Which states :

For AVR boards "Export compiled binary" generates hex.

If I understand correctly, Arduino Mega 2560 is AVR board ? Can I somehow still generate .bin ?

I've generated the .bin and downloaded it, but Arduino restarts after the download, to be exact, in the line before the while. I have 4kB of available memory, and the .bin is below 2kB.

vukovic:
I've generated the .bin and downloaded it, but Arduino restarts after the download, to be exact, in the line before the while. I have 4kB of available memory, and the .bin is below 2kB.

allocate the memory

char binaryUpdate[2000];

Hi everyone,

I have a sparkfun atsamd21 r3 breakout and added a sim800l module. I try to update my .bin file via gprs. I

download arduinoOTA library from github. I review all examples in it. All examples are using with wifi or ethernet

libraries except advanced. I read all topics carefully and Juraj's answers.

I can create .bin file by using "export compiled binary from Sketch menu" and put it server succesfully.

Juraj:
the library is for upload from IDE. but the InternalStorage object of WiFI101OTA library can be 'feed' from any source.

If I understand correctly, i have to use InternalStorage object for this process.

So i tried to add some parts of OTASketchDownload.ino in my code and remove one line of this sketch.

i removed this line. because sim 800l dont have any mac adress.

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

I did not remove anything

and i change these constants.

const char* SERVER = "my server ip";
const unsigned short SERVER_PORT = my port number;
const char* PATH = "/update.bin";
const unsigned long CHECK_INTERVAL = 5000;

I think that i can connect server succesfully but i could not update .bin file from server. Because

Update Status Code:-2 and nothing change in my atsamd21 board.

Where am i making a mistake?

Thanks for reply..

msg_comport.PNG

fth_41:
I think that i can connect server succesfully but i could not update .bin file from server. Because

Update Status Code:-2 and nothing change in my atsamd21 board.

Where am i making a mistake?

you have to code the download of the bin file over the GSM module. it is different then the download with Ethernet library. maybe TinyGSM library could work very similar as the Ethernet library to download the bin.

Hi again Juraj,

I downloaded TinyGSM library and checked the examples. After that i added my code some parts of httpClient example.

I think, I passed one step more successfully. I attached an image.

Now i am trying to do OTA process using your library (ArduinoOTA-->OTASketchDownload). But still could not.I added your ArduinoOTA library my code and edit handleSketchDownload() fuction.

A litte bit confused because when i add ArduinoOTA without "ethernet.h" i cant compile the code.

fth_41:
Hi again Juraj,

I downloaded TinyGSM library and checked the examples. After that i added my code some parts of httpClient example.

I think, I passed one step more successfully. I attached an image.

Now i am trying to do OTA process using your library (ArduinoOTA-->OTASketchDownload). But still could not.I added your ArduinoOTA library my code and edit handleSketchDownload() fuction.

A litte bit confused because when i add ArduinoOTA without "ethernet.h" i cant compile the code.

did you change EthernetClient to TinyGsmClient

Juraj:
did you change EthernetClient to TinyGsmClient

please correct me if i am wrong. I changed these in my code..

TinyGsmClient transport;

I removed these lines..

.
.
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
.
.
.

Serial.println("Initialize Ethernet with DHCP:");
if (Ethernet.begin(mac) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
while (true);
}
Serial.println("Ethernet connected");

Still I could not. So i looked ArduinoOTA.h.

Do I change these lines???

#if defined(ethernet_h_) || defined(ethernet_h) // Ethernet library
#ifdef NO_OTA_PORT
ArduinoOTAClass <EthernetServer, EthernetClient> ArduinoOTA;
#else
ArduinoOTAMdnsClass <EthernetServer, EthernetClient, EthernetUDP> ArduinoOTA;
#endif

fth_41:
please correct me if i am wrong. I changed these in my code..

TinyGsmClient transport;

I removed these lines..

.
.
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
.
.
.

Serial.println("Initialize Ethernet with DHCP:");
if (Ethernet.begin(mac) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
while (true);
}
Serial.println("Ethernet connected");

Still I could not. So i looked ArduinoOTA.h.

Do I change these lines???

#if defined(ethernet_h_) || defined(ethernet_h) // Ethernet library
#ifdef NO_OTA_PORT
ArduinoOTAClass <EthernetServer, EthernetClient> ArduinoOTA;
#else
ArduinoOTAMdnsClass <EthernetServer, EthernetClient, EthernetUDP> ArduinoOTA;
#endif

no, you don't use the local server and OTA port