Memory layout for PROGMEM

[fastbike]

I'm coding a SAMD21 based board. The upload will contain both code and application config data. (Note this is NOT the chip config data.) The application config data will be declared using the PROGMEM directive aand the initial upload will contain default values for this.

e.g.

const byte configdata [6] PROGMEM = { 0x90, 0xA2, 0xDA, 0x0F, 0xE1, 0x85 };

At some stage after the node has been deployed I'd like to be able to overwrite the app config data using an across the wire protocol (see Omzlo: The NoCAN protocol specification). There are tools in this protocol to specify the starting address, length and payload of firmware (app config data) to upload.

What I need to know is the address of the memory block that is allocated by the compiler when the PROGMEM directive is specified so I can set the starting address correctly.

Is there documentation that covers this ?

[sterretje]

To my knowledge, a SAMD21 has no progmem. You can find a memory location by using the printf family and using the %p format specifier.

char buffer1[64];

int param1 = 1;
int param2 = 2;
void setup() {
  Serial.begin(115200);
  while (!Serial)
    ;

  sprintf(buffer1, "address of param1 = %p", &param1);
  Serial.println(buffer1);
  sprintf(buffer1, "address of param2 = %p", &param2);
  Serial.println(buffer1);
}

void loop() {
  // put your main code here, to run repeatedly:
}

Output:

address of param1 = 0x10e
address of param2 = 0x10c

I'm not familiar with the SAMD21 but I'm reasonably sure that you can emulate EEPOM in flash; it might be a far easier solution. The flow would be something like

1. Verify (emulated) EEPROM CRC
2. If CRC fails, use program defaults
3. If CRC success, use EEPROM values

You can store your parameters in emulated EEPROM and update those.

[fastbike]

Interesting the compiler did not complain or emit any warnings with the code declaration

const byte configdata [6] PROGMEM = { 0x90, 0xA2, 0xDA, 0x0F, 0xE1, 0x85 };

I'll follow up on your suggestions to see what it turns up. Maybe also time to read the datasheet

[Koepel]

It has no PROGMEM, all you need is a 'const' variable and it will be stored in Flash.
The 'PROGMEM' keyword is added to be compatible, but it is an empty keyword.

It has also no EEPROM, but you can mimic the EEPROM behaviour with a library that uses Flash memory. A few other processor can do that same trick. But it is a trick.

The ATmega microcontroller is optimized a lot. As a result it can not execute code from ram and it can not read data from Flash. There is no data-pointer that can point to data in Flash. With special instructions it is possible to read data from Flash and the compiler can be told that a pointer should be treated as a pointer to Flash memory, so with a detour or two, the PROGMEM became possible.

[sterretje]

@fastbike

Interesting the compiler did not complain or emit any warnings with the code declaration

const byte configdata [6] PROGMEM = { 0x90, 0xA2, 0xDA, 0x0F, 0xE1, 0x85 };

For compatibility in Arduino land, PROGMEM is defined

#define PROGMEM

It's just empty so your line looks like below to the compiler

const byte configdata [6] = { 0x90, 0xA2, 0xDA, 0x0F, 0xE1, 0x85 };

File: C:\Users\sterretje\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino\api\deprecated-avr-comp\avr\pgmspace.h

[fastbike]

So if I declare a lump of const memory, is there any way I can definitively determine where the compiler will store that, or do I need to do something a bit more clever and have a function that would return the address of the start of the memory address, given that the optimising compiler could move it between version of my program.
Sort of building on your first answer sterretje

[fastbike]

And I'll follow up the emulated EEPROM

EEPROM Emulation with Arduino API

[Koepel]

Be clever and use 'const'.
Don't over-think it or try to over-clever it, that will mess it up.

Make a block of data and look at the compiler output.
With 'const':

Sketch uses 29368 bytes (11%) of program storage space.
Global variables use 2400 bytes (7%) of dynamic memory

Without 'const'

Sketch uses 29376 bytes (11%) of program storage space.
Global variables use 20464 bytes (62%) of dynamic memory

Without 'const', the data is still in Flash, because the data has to be somewhere. During startup, it is copied to a global variable, and the SRAM usage goes from 2400 bytes to 20464 bytes.

For a processor, it is common that a 'const' goes into Flash memory and nowhere else. That is the rule. That is what the 'const' keyword is supposed to do. The ATmega is the exception because it has no pointer to 'const' data in Flash as I wrote before, so that is where the trickery stuff is used, but it is not common.

[fastbike]

Thanks Koepel

For a processor, it is common that a 'const' goes into Flash memory and nowhere else. That is the rule. That is what the 'const' keyword is supposed to do.

That's got me on the right track (I'm new to this level of programming). I'll follow up by figuring out where my const block gets stored in the hex file and can work things out from there.
This post looks useful

Learning about the hex file

Hi folks! I am trying to learn more about the hex file that is generated when I compile a sketch. So down below, you can see a regular BLINK sketch and its hex file. Does every line on the hex file correspond to a certain command in the sketch ( like Void, pinMode, loop)? What are the uses of having a hex file? The last line in the hex file is ":00000001FF", which is 511 in decimal. Is there something special about this line becuase it seems that almost all sketches end with line. Ty for th…

[sterretje]

@fastbike

This post looks useful

That's for AVR processors; you'll have to find something for the SAMD.

Another problem is, if I'm not mistaken, that the result of avr-objdump is before the linker optimises the code; but I might be wrong with that.

[fastbike]

Good catch. I've opened the HEX output and can see the structure appears to conform to the Intel Hex format.
I'll dig into the SAMD stuff

[sterretje]

@fastbike

I'll dig into the SAMD stuff

I hope that that is the emulated EEPROM stuff, else I think that you're wasting your time. Don't get me wrong, it's always nice to learn things but if you want to get your project finished ...

[fastbike]

@sterretje

I hope that that is the emulated EEPROM stuff, else I think that you're wasting your time. Don't get me wrong, it's always nice to learn things but if you want to get your project finished ...

Thanks for your help, I've learnt a lot in the last few hours and no doubt I'm only starting to climb this wall.
I agree, it's good to learn new things, but don't forget the finish line. I often have to remind my team of this at work - hit a road block so just put a "mock" solution in to get past that block while you get the rest of the project working. You can always come back to put the correct solution in place as needed

[fastbike]

So things have evolved I'm using a bootloader and an external protocol which enables the "application" code to be reflashed over a communications network.

The bootloader currently puts the application code at address 0x2000. I could change this to put the code at a higher address, say 0x2400 which would free up 1k bytes of flash to store configuration data (this block will not be overwritten if a new application hex is uploaded).

So the problem now becomes how to get the Arduino code to access memory at a negative offset, some typical of virtual mapping perhaps ?

[Koepel]

That is a lot of effort, is uploading the configuration data together with the sketch so bad ?

There is no such thing as a negative offset, there are pointers that point to a memory location, that's all.

What I mean is, a pointer can not be negative: char *p = -0x100;

[fastbike]

Yeah, with 100 odd nodes across the network having to recompile code for each one to change the configuration is not an option. As well as being too error prone.

I'll be writing a separate PC utility that will provide a GUI for configuration of each device, outputting the data in a bytewise format that can then be uploaded across the network to targetted devices.

So the issue to be solved here is to provide an emulation of EEPROM reading that reads from the memory at 0x2000. I can handle the writing of that memory via the network protocol. So obviously not a pointer but a lower level assembly instruction then.

[fastbike]

This page looks interesting, providing functionality to access memory at specified locations
https://developer.arm.com/documentation/ka002853/latest

Determining whether I can use it via the Arduino eco system is my next task.

[Koepel]

Arduino is just a layer of software. Everything below it can be used, every C++, C and assembly code you want.
If the normal way would be "int var __attribute__((at(0x20000000)));" then that can be used in Arduino. I see most of the time that the compiler is told about a "section" at a certain location. There is a note about the GCC compiler and you need a pointer to Flash memory. Let us know if it works.

[fastbike]

Great. I'll take a look at doing this later today.

[fastbike]

So my code to define the configuration data will look something like ...

// use following to stash config data at a particular location
typedef struct {
  byte channel;
  byte id;
} config_test;

const config_test configData[2] __attribute__((at(0x2000))) = {{10, 1}, {11, 0}};

and the code will be accessed during the execution of the program

  int _delay =   configData[0].channel;

The board definition linker_scripts flash_with_bootloader.ld file that defines the offset for the application code will need to be amended as well, so the HEX file contains the correct address.

Time to take a look at the bootloader because that starts the application, and it expects to find it at 0x2000, rather than the new location of 0x2400