To work, your sketch must consume less than half of the available program memory. This allows you to temporarily write your entire sketch to the free flash memory, and then use optiboot_copy to copy the sketch from the temporary location to the correct address.
When the copy is done, obviously the return address in the call stack is incorrect. Hence the last lines of optiboot_copy clear the stack and jump to a user-specified address. In my case, that address is 0 which reboots the device.
If your sketch is larger than half the available program memory, in theory you could use this auto update mechanism to flash a small temporary program to the device, and use that smaller program to receive a larger program. Or you could extend this to work with some other form of temporary memory (ie. an SD card).
Why did I do this? I have multiple (currently 5) Pro Mini's connected with a proprietary network similar to I2C to a Raspberry Pi. They are all running the same firmware and I want to be able to update them without having to fiddle around with FTDI cables. For me, it works like this:
The Pi broadcasts off the new program 16 bytes at a time
The Pro Minis receive this and write it to a temporary memory buffer
When one full flash page is in memory, a checksum is verified to ensure no packets were lost
optiboot_page_fill and optiboot_page_write are then called to write the page to high flash
When the whole sketch is in high flash, optiboot_copy is used to copy it down to the proper address and then reboot
I have tested this with a Pro Mini with ATMEGA 328P and it works well for me. It will probably require slight modifications for other devices.
Unfortunately, these changes end up increasing the size of optiboot to 598 bytes, so it must consume 2 pages (1024 bytes) instead of just 1 page (512 bytes), but this is a price I was willing to pay.
westfw:
The latest optiboot github repository includes a version of the do_spm() code
but we would like to have a function to copy n pages from address x to address 0x0, to avoid "having code just below bootloader in NRWW section". and then reset please. too much?
I extended the WiFi101OTA library for SAMD to general OTA (or OTEthernet) upload library. I made it work with Ethernet, WiFiNina, WiFiLink, UIPEthernet and WiFiSpi. And I added nRF51 support. And I plan STM32F102 support. But I would like to support AVR ATmega too. I work on SDStorage option now, but for the InternalStorage option I would like to avoid the "code just below bootloader in NRWW section".
these changes end up increasing the size of optiboot to 598 bytes
Oh; I hadn't read carefully enough to notice that you wanted the whole copy loop in the bootloader.
Did you put it under the "BIGBOOT" conditional?
I'm not inclined to increase the "plain" optiboot bootloader image by ~100 bytes...
Adding additional vectors to the entry point seems like worrisome precedent
I wouldn't mind at all having the change submitted as a "pull request" - even when those aren't "accepted", they can be a useful reference for other people who want similar changes...
@Juraj It works for me, and I'm glad it saved you time. Hopefully it will work for you too.
The sketch on my Pro Minis is fairly basic, so unfortunately posting a portion of it won't help you. They are simple slave devices which accept basic commands from the master which is a Raspberry Pi.
However, I have attached "UploadFirmware.cpp" to the original post. This is the code which runs on the Raspberry Pi to perform the upload over my custom protocol similar to I2C. The RemoteXXX methods are the calls which talk to the Arduinos, and I have omitted that code since you will probably already have some other communication mechanism in place.
UploadFirmware.cpp:
Loads a compiled .hex file into memory in SPM_PAGESIZE blocks.
Uploads non-empty pages to my temporary memory buffer on the Pro Minis.
Performs a checksum to ensure the whole page was uploaded, and then flashes it to program memory, starting at address 0x7C00 and working its way down.
Once all pages are uploaded, it calls optiboot_copy to copy everything to the proper addresses and reboot.
@westfw Unfortunately I didn't think to put my changes under the BIGBOOT conditional, however I agree they certainly belong there.
I believe adding an additional vector is the easiest way to call do_spm_copy. In theory, it could somehow be implemented as another command type to do_spm, but that would require an if condition to be added to do_spm which would probably increase the boot loader size by more. I also require more arguments than do_spm.
I agree do_spm_copy is implemented as a bunch of calls to do_spm, and hence could in theory reside in regular program memory, but you obviously can't run code from a page which is being updated to different code, hence the loop has to live in the boot loader.
Now that I know there is interest, I will work on a moving my changes under BIGBOOT and submit them as a pull request for others to have access.
gmattinson:
Now that I know there is interest, I will work on a moving my changes under BIGBOOT and submit them as a pull request for others to have access.
please do it.
the Arduino OTA support for AVR boards can only work with boards with BIGBOOT (over 64 kB memory), because the networking library takes more then 50% of the flash for Uno, Leonardo and co.
I also created a very simple example sketch which is included in the pull request.
I worked on this yesterday. Your code works only for address below 64 kB. So you should remove the PR.
Making it work for 32 bit addresses turned out to be complicated. pgm_read_word_far doesn't work in bootloader. I don't know why. I continue with this today.
OP - if I were doing this, I'd be tempted to do it totally differently: I'd write a bootloader that works over your protocol, and when the device received a command to update itself, reset into the bootloader, let it do it's thing. On a chip with BOOTRST, this seems like it would be pretty solid. Maybe I'd be tempted to have it first write over the first page with a temporary one where the reset vector just pointed back to the bootloader, and then after everything else was written, then write the "real" first page of the flash...
Your code works only for address below 64 kB. So you should remove the PR.
Nah; It's fine for pull requests to sit around, with objection, forever. There's no downside, really.
Shucks, it can get merged even if it doesn't work everywhere... (at which point "doesn't work >64k" becomes an official "issue." I'm very proud that Optiboot has more issues than words of code! Documenting things is important, and using a bug-tracking capabilities like github's "issues" is one of the easiest ways to do so.
Thank you for figuring this out and submitting the pull request!
...and/or a branch can be created for the feature. Those who want the feature just use that branch. Everyone else continues using master. Pulling master into that branch at milestones keeps that branch up-to-date and keeps the feature out of master.
@Juraj, sorry it doesn't work for addresses > 64k. I only have 328p's, so unfortunately I can't help you try to fix this. If you figure it out, I would be happy to modify my pull request.
However, I would like to point out optiboot.c has a read_mem function which calls elpm inside a RAMPZ define, hence I believe you could write your own pgm_read_word_far to get this working fairly easily.
Snippets from optiboot.c which I think are required to create pgm_read_word_far:
if (newAddress & 0x8000) {
RAMPZ |= 0x01;
}
else {
RAMPZ &= 0xFE;
}
RAMPZ = (RAMPZ & 0x01) | ((getch() << 1) & 0xff); // get address and put it in RAMPZ
not the pgm_read_word_far is the problem. I test on atmega2560 the original example test_dospm.ino modified for 32 bit addresses. the example works for pages below flash address 0x1FC00 and crashes for address 0x1FC00 and above. I must burn the bootloader to make do_smp work again after the crash
