Flashing Sketches from SD card

I have built a Shield that plugs into the DUE board and runs off of a 3.6 VDC battery pack. The shield uses a DS3231 RTC chip to power up and control several 2 amp power relays, a DC/DC converter for +5VDC, and the shield includes an SD card slot, a BlueTooth Module, a xBee footprint, a cell modem, and it can FTP files up and down from FTP sites. The shield sleeps the entire assembly at 16 uAmps until the timer wakes it, then it follows a command file on the SD card for instructions for the DUE current program. I would like to download new sketches to the SD card from the FTP site and when the DUE is powered up from the RTC timer it would update the DUE’s sketch. Seems like all I would need is the bootloader to include the SD library (Chipselect is pin 53) and this would work. After reading through all of the forum I find a lot of people getting close to something like this, but no joy! Is there anyone close on this that I can collaborate with? Attached are pictures of the Shield and Due assembly:

There is no bootloader on the DUE. Nothing you can access and modify.

If it’s already reading a command script on the SD card, then why do you need to change the program in flash? Just make sure your commands can cover all imaginable future situations.

Of course that’s impossible to predict in advance, but one service visit per year should be feasible.

A JVM might be feasible… A quick google:
http://haiku-vm.sourceforge.net/

It is true that there is no bootloader on a Due. However, you can still write one. I have a CAN bus bootloader that I wrote. The general idea is that I store the CAN bootloader in the upper 256k FLASH segment and user code in the lower 256k segment. The reason for this is that the SAM3X cannot write to the same flash segment it is running from. You can work around this by making your bootloader all RAM resident functions but you are then limited to 96k for both your code and your RAM data.

I set the boot flag (there’s a flag to tell the processor which segment to boot from) to either Flash0 (user code) or Flash1 (bootloader). You can do this too. It halves the max size for your sketches but 256k is still quite a bit of room.

Here is what I’ve got:

There’s the user side code to set up for starting the bootloader and an example bootloader. Also included is instructions for how to setup Arduino IDE to work with these two things. Essentially what you’d have to do is switch from CAN to SDCard for where the bootloader gets its data from. This should not be that difficult and you’ve got 256k worth of code space to do it in so bootloader size is not a problem.

Or, if you’re brave try to do it all in RAM resident functions but keep in mind that every function call you use has to be in RAM. EVERY. ONE. Even standard library calls. This quickly gets tough to do. You’ll see that the bootloader does have RAM resident functions but very sparingly.

Mr. Collin80, I have written an extensive application for this shield I have built, but right now it is only occupying 17% of memory when I download it to the DUE via the USB. I can download binary and text files from my FTP site to the SD card on the shield (FAT32) so any time I need to update the DUE code I can push out a new image for all of the units to download and re-flash themselves with. The thing that makes this firmware upgradeablity a necessity is that the device I have built is going to be mounted on towers in the next few days, and before there are thousands of them on towers, I need to be able to remotely upgrade them. I do need to figure out what you've done here and how to get the SD.h library (and Wire.h?) in the upper 256K of flash along with your boot loader and then have it re-write the sketch in the lower 256K every time the code detects that a new sketch has been downloaded to the SD card. As soon as the new sketch starts, it would see that it is the same version number as the one on the SD card and delete the SD card Sketch file preventing endless re-programming. How do I get this done? I certainly would be glad to pay for someone's time to do it and I need it quickly, but I do think it should benefit the Arduino community and be open source after it's debugged and released. Please tell me how to proceed: The Arduino community benefits from commercial ventures like mine, and this shield I have built can certainly be used for many things (I expect to release a mature version of the hardware as well), and a SD boot loader for DUE would be an awesome thing for everyone and it would help bring Arduino forward as a viable product platform and not just the hobbyist bench. Can you help?

I am attaching to this post a top view of the Shield I have designed that features:
Cell Modem
Bluetooth Module
xBee Connector
MicroSD card
DS3231 Real Time Clock Calendar Chip with Alarm that powers up DUE Board (System sleeps at 16 uAmps from 3.6 VDC Battery pack)
Three Power Relays:
Relay #1 Powers DC/DC converter to provide 2 Amps +5 VDC for Arduino and Peripherals
Relay #2 Powers up Cell Modem and xBee with +5 VDC
Relay #3 Powers up aux board or peripherals with 2 Amps 3.6 VDC from Battery pack
Three Magnetic Reed Switches to power up and down the system and to wake up Bluetooth Module

TopSmall.jpg

I wish I could say that I've got the time to make this happen for you. But, I don't.

However, I can give a few more details about how you could do it.

The bootloader goes into the upper 256k by using a custom Arduino board definition that tells it that only the upper 256k exists so everything gets placed there. You can add whatever you want to that bootloader - the SD card routines and everything. The GitHub repo has a document that details how you can set up this custom board definition. You also need a way to combine two sketches together which is also explained.

The very general overview of how this process works: Your normal sketch runs in FLASH0 and starts by default. The bootloader is in FLASH1 and normally is not executed or used at all. Your sketch in Flash0 will contain a RAM resident function that configures the processor to boot from Flash1. It then forces a reset. Upon reset the bootloader in Flash1 runs instead of the user code in Flash0. The bootloader then does it's thing and calls the same type of RAM resident function to set the boot back to Flash0 and reboots again. In this way the user sketch can select the bootloader any time it wants and the bootloader can write a new sketch and automatically reset. It's about the best you can do on a SAM3X processor because it otherwise has only a ROM based bootloader that you thus cannot modify. As for how to ensure you don't endlessly boot loop - store the version of the sketch in your code and compare it. Perhaps even name firmware on the sdcard by version and then you can compare that number to the built-in version of the currently running sketch. You'd probably want to delete firmware images after you flash them.

It's a proven technique and will work fine but you'd have to modify everything to work with an SDCard. It shouldn't be too hard to do if you've already got remote connectivity and SDCard access. You should have all the resources to make it happen. The trickiest part is wrapping your head around how to compile the bootloader as a Flash1 executable and then flashing it into the proper place. Using a JTAG adapter does make it pretty easy. After you flash the bootloader up there you won't want to ever use the Arduino IDE upload functionality. It'd zap the bootloader. Once you go to using a bootloader you pretty much want to do your software upgrades via the bootloader only.

Mr. Collin80, can you tell me what JTAG programmer you would recommend? Do I need any of the Atmel IDE suites? Sounds like you are using the JTAG strait from the Arduino Sketch IDE. I have been using Visual Studio 2015 with vMicro plugin for Arduino. I don't know if it supports direct JTAG programming or not. Seems like there should be ORG statements for Arduino to the SAM that let you specify where a portion of code gets flashed in the Microcontroller program memory. I have written boot loaders in assembly before for MSP430 processors and understand the process, I'm just very unfamiliar with the SAM3X8E processor and I am still in the weeds learning the Arduino world. I am glad I found you and I am jumping into this project with all four feet now! Wish me luck!

For JTAG you can use a variety of hardware but the SAM-ICE is probably easiest to use and acquire. From there the Atmel Studio contains the tools to work with JTAG adapters. Arduino IDE uses bossac for programming. It doesn’t really have the concept of flashing in sections. But, you could probably grab the binary for the bootloader and the binary for the user sketch and copy them end to end then use bossac to flash it if you don’t want to use JTAG.

Hello,

Has any one successfully demonstrated the process explained by AdderD in post #6 ? Please point me of any.

Hello,

Same Question, Has any one successfully demonstrated the process explained by AdderD in post #6 ?

I can under stand how he explains this process. But my SD understanding is very weak.