SSLClient & ArduinoOTA

Hi,
I am using Arduino Zero 256kB with W5500. Firstly, I done with OTA successful firmware(89kB). Next, I also able to run MQTT with SSLClient (firmware 149kB). Now OTA cannot store because not enough space.

I try to figure out what maximum store is available using InternalStorage.debugPrint(); but it doesnt print anything. Did I miss something?

Is there any other solution to modify size storage or modifly SSLClient to make small size.

it prints to Serial. maybe you use SerialUSB?

from https://github.com/JAndrassy/ArduinoOTA/tree/master#installation

With InternalStorage the sketch binary size is limited to half of the available flash memory size. The available flash size may be reduced by the bootloader and EEPROM emulation space.

i did use #define Serial SerialUSB

do you any other suggestion to implement MQTTS with self-signed certificate

in InternalStorage.cpp?

No, before header in main code. Not works.

I edit on InternalStorage.cpp, its working now.

this is the output

16:44:59.356 -> Update status code: 200
16:44:59.403 -> Server returned update file of size 88404 bytes
16:44:59.403 -> SKETCH_START_ADDRESS 8192
16:44:59.403 -> PAGE_SIZE 64
16:44:59.403 -> MAX_FLASH 262144
16:44:59.403 -> MAX_PARTIONED_SKETCH_SIZE 126976
16:44:59.403 -> STORAGE_START_ADDRESS 135168

Hi, do you have any guide how to use/modify ArduinoOTA for external flash/eeprom?

for SAMD the library has SerialFlashStorage which can be used with the SFU library.
this is inherited from the WiFi101OTA library. I didn't test it.

I am try to compile SFU_Usage example with arduino zero, but show error

Compiling library "SFU"
"C:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4/bin/arm-none-eabi-g++" -mcpu=cortex-m0plus -mthumb -c -g -Os -w -std=gnu++11 -ffunction-sections -fdata-sections -fno-threadsafe-statics -nostdlib --param max-inline-insns-single=500 -fno-rtti -fno-exceptions -MMD -DF_CPU=48000000L -DARDUINO=10819 -DARDUINO_SAMD_ZERO -DARDUINO_ARCH_SAMD -D__SAMD21G18A__ -DUSB_VID=0x2341 -DUSB_PID=0x804d -DUSBCON "-DUSB_MANUFACTURER=\"Arduino LLC\"" "-DUSB_PRODUCT=\"Arduino Zero\"" "-IC:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0/CMSIS/Include/" "-IC:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0/CMSIS/Device/ATMEL/" "-IC:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.13\\cores\\arduino/api/deprecated" "-IC:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.13\\cores\\arduino/api/deprecated-avr-comp" "-IC:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.13\\cores\\arduino" "-IC:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.13\\variants\\arduino_zero" "-IC:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.13\\libraries\\SFU\\src" "C:\\Users\\SC\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.13\\libraries\\SFU\\src\\SFU.cpp" -o "C:\\Users\\SC\\AppData\\Local\\Temp\\arduino_build_221219\\libraries\\SFU\\SFU.cpp.o"
C:\Users\SC\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.13\libraries\SFU\src\SFU.cpp:42:4: error: #error "Unsupported board!"
   #error "Unsupported board!"
    ^~~~~
Using library SFU at version 1.0.0 in folder: C:\Users\SC\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.13\libraries\SFU 
exit status 1
Error compiling for board Arduino Zero (Native USB Port).

I use SFU_Usage example, if i try Load_binary example,

12:43:54.479 -> Mounting ... mount() failed with error code 
12:44:10.760 -> -10025

Why this happen? Also i try to understand what and how SFU.h can be use with external SPI Flash memory, but i cant figure it out, where is instance, method, or definition for this SFU.h library.

Right now, I have AT25EU (1Mbit-Renesas) and waiting for delivery W25Q64(64Mbit-WinBond).

it looks like the library only supports MKR boards. maybe because there is a shield with Serial Flash. the SFU library is for that shield.
for Zero you would have to modify the SFU library. (I know how)

EDIT: SFU library works together with Arduino_MKRMEM library

Okay, can you guide me or do you have reference how to modify SFU library?

first you have to make the Arduino_MKRMEM library work with the Zero

I will go through this library first.

Does this happen because it doesnt identify AT23EU?
Do I need to add AT25EU (1Mbit-Renesas) into Arduino_MKRMEM library?

Hi,

I just realize, Arduino Zero can work with Arduino_MKRMEM library. It just my flash is protected.
I can run all example from Arduino MKRMEM example success.

RawFlash Access example

16:50:12.467 -> ID: 1F 10 01
16:50:12.467 -> Erasing chip
16:50:12.467 -> Comparison OK
16:50:12.467 -> RD: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
16:50:12.508 -> Programming page
16:50:12.508 -> WR: 0 1 2 3 4 5 6 7 8 9 A B C D E F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F 40 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D 5E 5F 60 61 62 63 64 65 66 67 68 69 6A 6B 6C 6D 6E 6F 70 71 72 73 74 75 76 77 78 79 7A 7B 7C 7D 7E 7F 80 81 82 83 84 85 86 87 88 89 8A 8B 8C 8D 8E 8F 90 91 92 93 94 95 96 97 98 99 9A 9B 9C 9D 9E 9F A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 AA AB AC AD AE AF B0 B1 B2 B3 B4 B5 B6 B7 B8 B9 BA BB BC BD BE BF C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 CA CB CC CD CE CF D0 D1 D2 D3 D4 D5 D6 D7 D8 D9 DA DB DC DD DE DF E0 E1 E2 E3 E4 E5 E6 E7 E8 E9 EA EB EC ED EE EF F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 FA FB FC FD FE FF 
16:50:12.555 -> RD: 0 1 2 3 4 5 6 7 8 9 A B C D E F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F 40 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D 5E 5F 60 61 62 63 64 65 66 67 68 69 6A 6B 6C 6D 6E 6F 70 71 72 73 74 75 76 77 78 79 7A 7B 7C 7D 7E 7F 80 81 82 83 84 85 86 87 88 89 8A 8B 8C 8D 8E 8F 90 91 92 93 94 95 96 97 98 99 9A 9B 9C 9D 9E 9F A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 AA AB AC AD AE AF B0 B1 B2 B3 B4 B5 B6 B7 B8 B9 BA BB BC BD BE BF C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 CA CB CC CD CE CF D0 D1 D2 D3 D4 D5 D6 D7 D8 D9 DA DB DC DD DE DF E0 E1 E2 E3 E4 E5 E6 E7 E8 E9 EA EB EC ED EE EF F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 FA FB FC FD FE FF 
16:50:12.555 -> Comparison OK
16:50:12.555 -> Sector erase
16:50:12.601 -> RD: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
16:50:12.601 -> Comparison OK

So, what is the next step?

the SFU library is a wrapper for a binary of a compiled sketch which works as a second stage bootloader. it is this sketch
https://github.com/arduino/ArduinoCore-samd/blob/master/libraries/SFU/extras/SFUBoot/SFUBoot.ino
so make this sketch work with Zero.
prepare into the external flash a bin for Blink sketch with the current SFU library included. that way the Blink sketch will be suitable to be loaded with SFUBoot.ino

the following step will be to add your version of SDUBoot to SFU library with
https://github.com/arduino/ArduinoCore-samd/blob/master/libraries/SFU/extras/SFUBoot/build.sh

Hi,
In my Arduino IDE doesnt have

and

so, i download and save into example SFU directory.
Right now, I could not figure out how make SFU.h work with Zero. So that, I just use, mkrzero.h by
redefine SFU.cpp

#if defined(ARDUINO_SAMD_MKRZERO) || defined(ARDUINO_SAMD_ZERO)
  #include "boot/mkrzero.h"

Based on example from here. I rewrite to use with W5500 Ethernet.

/*

  This example downloads sketch update over NB/Cat1 network. (but can also be used for wifi / GSM / other types of networks as well)
  You can choose between HTTP and HTTPS connection.
  It downloads the binary file and saves it to the MKRMEM flash to store and apply the downloaded binary file.
  You can also works with other types of flash memory such as winbond W25QXX, just remember to adjust the flash page size if needed. (currently set to 256 bytes per page)

  To create the bin file for update of a SAMD board (except of M0),
  use in Arduino IDE command "Export compiled binary".
  To create a bin file for AVR boards see the instructions in README.MD.
  To try this example, you should have a web server where you put
  the binary update. (such as amazon s3)
  Modify the constants below to match your configuration.

  Important note:
  Don't forget to include the SFU library in the new sketch file as well!

  Created based on ArduinoOTA library in March 2023
  by Amir Pinkert
  based on Nicola Elia and Juraj Andrassy work

*/


#include <Ethernet.h>
#include <SPI.h>
#include <ArduinoHttpClient.h>
#include <SFU.h>
#include <Arduino_MKRMEM.h>

#define Serial SerialUSB

EthernetClient transport;

byte mac[6] = {0xDE,0xAD,0xBE,0xEF,0xFE,0x05};

const char* BIN_FILENAME = "UPDATE.BIN";

void setup() {
  
  Serial.begin(115200);
  while (!Serial);

  Ethernet.init (0);
  Ethernet.begin(mac);
  Serial.print("server is at ");
  Serial.println(Ethernet.localIP());
  flash.begin();

  /*
  // you can format the flash at this point, but everything stored on it will be deleted so use with care
  Serial.println("Erasing chip ...");
  flash.eraseChip();

  Serial.println("Mounting ...");
  int res = filesystem.mount();
  if(res != SPIFFS_OK && res != SPIFFS_ERR_NOT_A_FS) {
    Serial.println("mount() failed with error code "); Serial.println(res); return;
  }

  Serial.println("Unmounting ...");
  filesystem.unmount();

  Serial.println("Formatting ...");
  res = filesystem.format();
  if(res != SPIFFS_OK) {
    Serial.println("format() failed with error code "); Serial.println(res); return;
  }
  */


  
  Serial.print("Mounting ... ");
  if(SPIFFS_OK != filesystem.mount()) {
    Serial.println("mount() failed with error code "); Serial.println(filesystem.err()); return;
  }
  Serial.println("OK");


  Serial.print("Checking ... ");
  if(SPIFFS_OK != filesystem.check()) {
    Serial.println("check() failed with error code "); Serial.println(filesystem.err()); return;
  }
  Serial.println("OK");

  // now connect to network

  Serial.println("Starting Arduino web client.");

  // connection state
  boolean connected = false;

  // start the modem with NB.begin()

  // check for updates
  checkForOTAUpdates();

}

void loop() {


  // add your normal loop code below ...
}


void checkForOTAUpdates() {
  const char* SERVER = "iot.armscloud.com";  // Set your correct hostname
  const unsigned short SERVER_PORT = 80;                              // Commonly 80 (HTTP) | 443 (HTTPS)
  //const char* PATH = "/update/ap_lan_v1_ssl.bin";                                 // Set the URI to the .bin firmware
  //const char* PATH = "/update/SFUBoot.bin";  
  //const char* PATH = "/update/SFUBoot_boot.bin";  
  const char* PATH = "/update/Blink_sfu.bin";  
  //const char* PATH = "/update/Blink_sfu_boot.bin";  
  
  HttpClient client(transport, SERVER, SERVER_PORT);  // HTTP

  char buff[32];
  snprintf(buff, sizeof(buff), PATH);

  Serial.print("Check for update file ");
  Serial.println(buff);

  // Make the GET request
  client.get(buff);

  int statusCode = client.responseStatusCode();
  Serial.print("Update status code: ");
  Serial.println(statusCode);
  if (statusCode != 200) {
    client.stop();
    return;
  }

  unsigned long length = client.contentLength();
  if (length == HttpClient::kNoContentLengthHeader) {
    client.stop();
    Serial.println("Server didn't provide Content-length header. Can't continue with update.");
    return;
  }
  Serial.print("Server returned update file of size ");
  Serial.print(length);
  Serial.println(" bytes");


  Serial.println("Creating \"UPDATE.BIN\" ... ");
  File file = filesystem.open(BIN_FILENAME, CREATE | WRITE_ONLY| APPEND);
  Serial.println("File created");

  client.setTimeout(30000); // works well with 30 seconds

  byte b;
  uint16_t flashBufferSize = 256; // adjust according to your flash page size
  byte flashBuffer[flashBufferSize]; 
  uint16_t i = 0;

  int localChecksum = 0; // this is optional - you can get the intended checksum value from your server and compare with your local file to make sure your local file was downloaded correctly

  while (length > 0) {
    if (!client.readBytes(&b, 1)) // reading a byte with timeout
      break;

    // add to flashBuffer
    flashBuffer[i] = b;

    // add to checksum
    int bValue = (int)b;
    localChecksum = localChecksum + bValue;

    i ++;

    // check if flashBuffer is full
    if (i > (flashBufferSize - 1)){

      // write to file
      file.write((void *)flashBuffer, flashBufferSize);

      // reset i
      i = 0;

    }

    length--;


  }

  // add whats left on the flashBuffer
  if (i > 0){
  
    byte remainderBuffer[i];
    
    // fill buffer
    for (uint16_t x=0; x <= i; x++) {
      remainderBuffer[x] = flashBuffer[x];
    }

    // write to file
    file.write((void *)remainderBuffer, i);
  
  }


  file.close();
  client.stop();


  if (length > 0) {
    filesystem.remove(BIN_FILENAME);
    filesystem.unmount();
    Serial.print("Timeout downloading update file at ");
    Serial.print(length);
    Serial.println(" bytes. Can't continue with update.");
    return;
  }

  Serial.print("Local checksum:");
  Serial.println(localChecksum);

  Serial.print("Download finished. Unmounting ... ");
  filesystem.unmount();
  Serial.println("OK");

  Serial.println("Sketch update apply and reset.");

  Serial.flush();

}

Result:

14:53:18.339 -> server is at 192.168.0.214
14:53:18.339 -> Mounting ... OK
14:53:18.710 -> Checking ... OK
14:53:25.784 -> Starting Arduino web client.
14:53:25.784 -> Check for update file /update/Blink_sfu.bin
14:53:26.854 -> Update status code: 200
14:53:26.902 -> Server returned update file of size 46260 bytes
14:53:26.902 -> Creating "UPDATE.BIN" ... 
14:53:27.182 -> File created
14:53:29.926 -> Local checksum:3381931
14:53:29.926 -> Download finished. Unmounting ... OK
14:53:29.926 -> Sketch update apply and reset.

It doesnt update the Zero board.

I try with both link my Blink_sfu.bin and Blink_sfu_boot.bin

this is Blink.ino

#include <Arduino_MKRMEM.h>
#include <SFU.h>

// the setup function runs once when you press reset or power the board
void setup() {
  // initialize digital pin LED_BUILTIN as an output.
  pinMode(LED_BUILTIN, OUTPUT);
}

// the loop function runs over and over again forever
void loop() {
  digitalWrite(LED_BUILTIN, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);                       // wait for a second
  digitalWrite(LED_BUILTIN, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);                       // wait for a second
}

Is this the correct step?

Edit: Hardware used => Arduino Zero,W5500, W25Q64FV

does SFUBoot work as sketch? reading bin from prepared file on external flash?

  1. SFUBoot.ino is for Zero
    I upload SFUboot on Zero, then IDE doesnt detect the Zero anymore. I guess its doesnt have Zero bootloader anymore. (I reprogram Zero with Blink_withbootloader.hex)

  2. Blink.bin (included SFU.h) for external flash
    How do I put bin sketch into external flash, do I need to convert into hex and put into header file?

  3. build.sh
    How it work and use it?

SFUBoot is a normal Arduino sketch. why it would not work on Zero?

as you can see in SFUBoot it reads a file named UPDATE.BIN. use "Export compiled binary" from Sketch menu to get the bin file. I guess you have to write a sketch which stores the bin into the flash so that SFUBoot finds it. It is how https://github.com/JAndrassy/ArduinoOTA/blob/master/src/SerialFlashStorage.h does it.

build.sh is a bash script. It runs on Linux command line. I can run it for you once you have working SFUBoot.

Little bit confuse here, I should upload SFUBoot or SDUBoot sketch into Zero?

sorry SFUBoot of course