Writing to flash from the application

Hello,

I'm trying to write to the program flash from my application on an Arduino Uno but having no success. There's many posts on this topic out there on the net but also a lot of misinformation and no clear answers.

According to the "memory programming" section of the datasheet (section 27) there's two "lock bits" that control the behavior of the SPM and LPM instructions when executed in the application section. These lock bits are BLB01 and BLB02.

Table 27-3 shows the way the bits work, it says that in "BLB0 mode 1" (ie. BLB01=1 and BLB02=1) there's "No restrictions for SPM or LPM accessing the Application section."

I've looked at my lock bits using the LPM instruction and they're set to 0x0f, ie. BLB01=1 and BLB02=1. This 0x0f value also matches the lock bits defined in my "boards.txt" file, so that's exactly what they should be (ie. my Arduino is perfectly normal).

I should be able to write to the "application" area of flash memory, correct? I don't see why it should fail.

Here's some code that shouldn't blink for very long...

void setup() {
  Serial.begin(9600);
  pinMode(LED_BUILTIN, OUTPUT);
}

volatile int page = 0;
volatile byte lock = 0;
void loop() {
  digitalWrite(LED_BUILTIN, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(100);                       // wait for a second
  digitalWrite(LED_BUILTIN, LOW);    // turn the LED off by making the voltage LOW
  delay(100);                       // wait for a second

  asm("cli");
  // Read my lock bits
  asm("ldi ZH,0");
  asm("ldi ZL,1");
  asm("ldi r24,0x09");  // SELFPRGEN|BLBSET - Erase a page of flash
  asm("out 0x37,r24");  // SPMCSR
  asm("lpm");
  asm("sts lock,r0");
  
  // This should erase all the flash pages and bomb the program
  asm("ldi r24,9");     // SELFPRGEN|RWWSRE - re-enable the RWW section
  asm("out 0x37,r24");  // SPMCSR
  asm("spm");
  asm("nop");
  asm("nop");
  asm("ldi ZH,page");
  asm("ldi ZL,page+1");
  asm("ldi r24,3");     // SELFPRGEN|PGERS - Erase a page of flash
  asm("out 0x37,r24");  // SPMCSR
  asm("spm");
  asm("sei");
  page += 128;          // Flash pages are 128 bytes

  // Show the lock bits
  Serial.print("lock=0x");
  Serial.println(lock,HEX);
}

I've never gone to to that level, but this may help:

https://forum.arduino.cc/index.php?topic=332191.0

In AVR, apparently, only code which is running in the context of the bootloader can issue the instructions to write to flash memory.

to write to flash you need Optiboot 7 as bootloader and this example https://github.com/Optiboot/optiboot/tree/master/optiboot/examples/test_dospm

6v6gt: I've never gone to to that level, but this may help:

https://forum.arduino.cc/index.php?topic=332191.0

In AVR, apparently, only code which is running in the context of the bootloader can issue the instructions to write to flash memory.

Permissions are controlled by the fuse bits. The way I read the datasheet it should work, only the bootloader area is protected (which is exactly as it should be, if I choose to overwrite my own program then that's my problem and nobody else's...)

Juraj: to write to flash you need Optiboot 7 as bootloader and this example https://github.com/Optiboot/optiboot/tree/master/optiboot/examples/test_dospm

The thing is... I want to distribute this so other people can use it. I don't want to force particular bootloaders on people, not many will have the resources to update theirs.

Having said that, the Arduino I'm using is very old. I might try a few more and see what happens.

fungus: Permissions are controlled by the fuse bits. The way I read the datasheet it should work, only the bootloader area is protected (which is exactly as it should be, if I choose to overwrite my own program then that's my problem and nobody else's...)

the flash on ATmega328(p) can be written only by code running in bootloader section. bootloader section is limited to last 4 kB of flash

Juraj: the flash on ATmega328(p) can be written only by code running in bootloader section. bootloader section is limited to last 4 kB of flash

And the optiboot bootloader takes onlt 512b of flash, which is typically what the bootloader partition is set to; hence why you need to use the versions of optiboot with that feature in order to write to flash from the application (it calls out to bootloader code)

Juraj: the flash on ATmega328(p) can be written only by code running in bootloader section. bootloader section is limited to last 4 kB of flash

Yes, I just found this line in the middle of the datasheet:

6.3.1 Application Section

"The Application section is the section of the Flash that is used for storing the application code ... the SPM instruction is disabled when executed from the Application section."

So it looks as if my great idea is a non-starter.

My lock bits are set to "0x0f" so I can't even read the bootloader area to look for hacks, I have to assume that a lot of other people's are, too.

If you bootload with MiniCore which has latest version of optiboot, you would be able to write flash from application (see the docs for that core - i think an example is linked somewhere in his documentation)

Just to add a link to what DrAzzy said: https://github.com/MCUdude/MiniCore#write-to-own-flash

fungus: Having said that, the Arduino I'm using is very old. I might try a few more and see what happens.

I doubt you'll find any Arduino board with the modern Optiboot bootloader on it. Well, this one will have it: https://www.tindie.com/products/MCUdude/dip-40-arduino-compatible-development-board/ but certainly none of the standard official Arduino boards or their clones and derivatives will have it.

pert: I doubt you'll find any Arduino board with the modern Optiboot bootloader on it. certainly none of the standard official Arduino boards or their clones and derivatives will have it.

So basically it's a showstopper. :'(

fungus: So basically it's a showstopper. :'(

Does it have to be AVR ? What is the application or product you are developing ?

fungus: So basically it's a showstopper. :'(

Not at all - just get out your ISP programmer and bootload with MiniCore to get modern optiboot and you're in business. Once you've done it a few times, it's 2 minutes, maybe 5 if you're using arduino as isp and have to wire both sides. IMO best practice for power user is to bootload every '328p-based board with MiniCore's optiboot anyway, it's strictly better than the alternatives, and then they all use the same board menu option.

I use USBAsp as my ISP programmer of choice. Available on ebay for insanely low prices (be sure to get the 10p-6p adapter). I have also had decent results with USBTinyISP (there's a nice purple board with 6p cable), and have a nano with ArduinoISP on it and the relevant pins hardwired to a 6p connector with the solder joints covered in hotglue for strain relief. All work well, though I generally reach for the USBAsp unless there's some reason not to.

fungus: Hello,

I'm trying to write to the program flash from my application on an Arduino Uno but having no success. There's many posts on this topic out there on the net but also a lot of misinformation and no clear answers.

According to the "memory programming" section of the datasheet (section 27) there's two "lock bits" that control the behavior of the SPM and LPM instructions when executed in the application section. These lock bits are BLB01 and BLB02.

Table 27-3 shows the way the bits work, it says that in "BLB0 mode 1" (ie. BLB01=1 and BLB02=1) there's "No restrictions for SPM or LPM accessing the Application section."

I've looked at my lock bits using the LPM instruction and they're set to 0x0f, ie. BLB01=1 and BLB02=1. This 0x0f value also matches the lock bits defined in my "boards.txt" file, so that's exactly what they should be (ie. my Arduino is perfectly normal).

I should be able to write to the "application" area of flash memory, correct? I don't see why it should fail.

Here's some code that shouldn't blink for very long...

void setup() {
  Serial.begin(9600);
  pinMode(LED_BUILTIN, OUTPUT);
}

volatile int page = 0; volatile byte lock = 0; void loop() {  digitalWrite(LED_BUILTIN, HIGH);   // turn the LED on (HIGH is the voltage level)  delay(100);                       // wait for a second  digitalWrite(LED_BUILTIN, LOW);    // turn the LED off by making the voltage LOW  delay(100);                       // wait for a second

 asm("cli");  // Read my lock bits  asm("ldi ZH,0");  asm("ldi ZL,1");  asm("ldi r24,0x09");  // SELFPRGEN|BLBSET - Erase a page of flash  asm("out 0x37,r24");  // SPMCSR  asm("lpm");  asm("sts lock,r0");    // This should erase all the flash pages and bomb the program  asm("ldi r24,9");     // SELFPRGEN|RWWSRE - re-enable the RWW section  asm("out 0x37,r24");  // SPMCSR  asm("spm");  asm("nop");  asm("nop");  asm("ldi ZH,page");  asm("ldi ZL,page+1");  asm("ldi r24,3");     // SELFPRGEN|PGERS - Erase a page of flash  asm("out 0x37,r24");  // SPMCSR  asm("spm");  asm("sei");  page += 128;          // Flash pages are 128 bytes

 // Show the lock bits  Serial.print("lock=0x");  Serial.println(lock,HEX); }

Code to write to the application section needs to be located in the bootloader section. Your asm code is in "loop()" which is, of course, in the application section. Hence it cannot work. And, no you do not need a special bootloader (or a bootloader at all) to make it work. Read the datahseet again.

fungus: Permissions are controlled by the fuse bits. The way I read the datasheet it should work, only the bootloader area is protected (which is exactly as it should be, if I choose to overwrite my own program then that's my problem and nobody else's...)

Read the datasheet again.

fungus: Yes, I just found this line in the middle of the datasheet:

6.3.1 Application Section

"The Application section is the section of the Flash that is used for storing the application code ... the SPM instruction is disabled when executed from the Application section."

So it looks as if my great idea is a non-starter.

My lock bits are set to "0x0f" so I can't even read the bootloader area to look for hacks, I have to assume that a lot of other people's are, too.

Your "great idea" is indeed a great idea. Just put your code that does the actual writing of flash IN THE BOOTLOADER SECTION and call it from your application and it will work.

It may help to realize that the flash section "circuitry" doesn't work "normally" while the actual writing is taking place.

How can flash write code running in the application section work when a write to the application section "pulls the rug out from under" your code during the write?

In other words, you need to be "there" to write "here" and you need to be "here" when you write "there".

If you are "here" and try to write "here", you would simply trash your own "here" (which is why the AVR disallows it).

Make sense?

DrAzzy: Not at all - just get out your ISP programmer and bootload with MiniCore to get modern optiboot and you're in business. Once you've done it a few times, it's 2 minutes, maybe 5 if you're using arduino as isp and have to wire both sides. IMO best practice for power user is to bootload every '328p-based board with MiniCore's optiboot anyway, it's strictly better than the alternatives, and then they all use the same board menu option.

IMOPO, the best way for a "power user" is to program the AVR with an ISP each time and not even bother with a bootloader.

And this with code written with a text editor and compiled with AVR-GCC and a makefile using "void main (void)" instead of the ridiculous "setup" and "loop" nonsense.

As a bonus you get to play with the entire bootloader section memory space and use it for more than blinking pin 13.

@krupski

IMOPO, the best way for a "power user" is to program the AVR with an ISP each time and not even bother with a bootloader.

I have to disagree a bit. Serial uploading is more comfortable, faster and without need of additional device. If you are developing or tune the APP it is really useful. So, I do not think it is not for "power user". 512B or even 1kB of memory is really small tax for the bootloader and if your program need almost whole flash, it's time to consider different chip or different programming approach or do some cuts in code.

Anyway, I have highly appreciate your knowledge, specifically in problem related to current topic. I must say I am always reading your posts with interest (karma++).

@krupski

Code to write to the application section needs to be located in the bootloader section.

Need to be out of target region for write, not in the boot region exactly. Does it?

Budvar10: @krupskiNeed to be out of target region for write, not in the boot region exactly. Does it?

Well, the AVR enforces the separation of the boot and application sections, but assuming that it DIDN'T, I am not sure if the flash writing circuitry would disable a running program from running in the same SECTION but a different AREA or not.

That is, say your code was in the application section at address 0x1000 and you tried to write to flash in the application section at address 0x7000 (i.e well outside the address space of your code), I don't know if it would work or not.

Of course, the question is moot since the AVR disallows actions like that which makes me assume that the entire section is unable to be read during a write operation. Otherwise, the enforced separation of sections would disable a very useful ability!

I used to play with Motorola 68HC11 microcontrollers and I know that trying to program internal EEPROM could not be done with code IN eeprom because putting the eeprom into write mode made it temporarily unable to read (i.e. trying to program it with code inside it would "pull the rug out from under" the running code).

BTW the 'HC11 did not have any hardware enforcement of being in one area while programming another, it just worked that way.