F() macro and sprintf

Hi everybody

Edit:
Platform is Arduino MKR Wifi 1010

It was my understanding that you cannot pass __FlashStringHelper type arguments to the variable-length argument list of sprintf().

And it was my understanding that one should use sprintf_P() and the (capitalized) %S format-identifier for Flash-located strings (const char*).

My experimenting and the code below however are telling a different story.
What am I not understanding here??

  char buf[256];
  snprintf(buf,255,"snprintf-s: %s",F("PROGMEM-STRING"));
  Serial.println(buf);
  
  snprintf(buf,255,"snprintf-S: %S",F("PROGMEM-STRING"));
  Serial.println(buf);

  snprintf_P(buf,255,"snprintf_P-s: %s",F("PROGMEM-STRING"));
  Serial.println(buf);
  
  snprintf_P(buf,255,"snprintf_P-S: %S",F("PROGMEM-STRING"));
  Serial.println(buf);

Output:
snprintf-s: PROGMEM-STRING
snprintf-S: S
snprintf_P-s: PROGMEM-STRING
snprintf_P-S: S

warning: format '%S' expects argument of type 'wchar_t*', but argument 4 has type 'const char*'

so "%S" is not related to PROGMEM in any way.

snprintf_P expects the format string in PROGMEM.

The optimizer will probably make some strange combinations work.

No one could show a small demonstration sketch that I asked for here: https://forum.arduino.cc/t/resolved-without-hacks-how-i-used-f-flash-memory-strings-in-snprintf-etc-for-arduino-nano/891927/14
And I could not make it work myself.

So if no one can give some prove, then Whandall is right and the others are wrong and it is wrong information or fake or bad documentation.

The Arduino MKR Wifi 1010 does not need PROGMEM. I think it is enough if you make a string a 'const'.

1 Like

so it is

The %S/PROGMEM is a feature of avr libc so it only pertains to avr processors.

https://www.nongnu.org/avr-libc/user-manual/group__avr__stdio.html#gaa3b98c0d17b35642c0f3e4649092b9f1

  • S Similar to the s format, except the pointer is expected to point to a program-memory (ROM) string instead of a RAM string.

That message was generated while compiling for a Nano.

I just looked at the avr libc sources for vfprintf.c and it has support for %S->PROGMEM string.

http://cvs.savannah.nongnu.org/viewvc/avr-libc/avr-libc/libc/stdio/vfprintf.c?view=log

It looks like %S->wchar got added to the gcc compiler and conflicts with the avr-libc use of %S->PROGMEM string.

The Arduino MKR Wifi 1010 does not need PROGMEM. I think it is enough if you make a string a 'const'.

So you are saying, all const char *ptr are pointing into program memory?

The optimizer will probably make some strange combinations work.

Two questions:
Is it possible to prevent the optimizer from doing its magic?

How will I ever know if am using my pointers correctly, if even const char* blobs are residing in program memory (as Koepel claimed)?

Interesting... Looks like the 'wchar' warning only happens for snprintf(), not snprintf_P().

Also, the warnings do not prevent the compiler from generating the right code. When the pointer's address space matches the format string, the string is displayed correctly. When the pointer's address space does not match the format string, the string is not displayed correctly.

char buf[256];

void setup()
{
  Serial.begin(115200);
  delay(200);

  Serial.println("Valid pointers:");

  snprintf(buf, 255, "snprintf-s: %s", "NORMAL-STRING");
  Serial.println(buf);

  snprintf(buf, 255, "snprintf-S: %S", PSTR("PSTR-STRING"));
  Serial.println(buf);

  snprintf(buf, 255, "snprintf-S: %S", (const char *) F("F_MACRO-STRING"));
  Serial.println(buf);

  snprintf_P(buf, 255, PSTR("snprintf_P-s: %s"), "NORMAL-STRING");
  Serial.println(buf);

  snprintf_P(buf, 255, PSTR("snprintf_P-S: %S"), PSTR("PSTR-STRING"));
  Serial.println(buf);

  snprintf_P(buf, 255, (const char *) F("snprintf_P-S: %S"), (const char *) F("F_MACRO-STRING"));
  Serial.println(buf);

  Serial.println("Invalid pointers:");

  snprintf(buf, 255, "snprintf-s: %s", PSTR("PSTR-STRING"));
  Serial.println(buf);

  snprintf(buf, 255, "snprintf-s: %s", (const char *) F("F_MACRO-STRING"));
  Serial.println(buf);

  snprintf(buf, 255, "snprintf-S: %S", "NORMAL-STRING");
  Serial.println(buf);
  

  snprintf_P(buf, 255, PSTR("snprintf_P-s: %s"), PSTR("PSTR-STRING"));
  Serial.println(buf);
  
  snprintf_P(buf, 255, PSTR("snprintf_P-s: %s"), (const char *) F("F_MACRO-STRING"));
  Serial.println(buf);

  snprintf_P(buf, 255, PSTR("snprintf_P-S: %S"), "NORMAL-STRING");
  Serial.println(buf);
}

void loop() {}

Run on an Arduino UNO:

Valid pointers:
snprintf-s: NORMAL-STRING
snprintf-S: PSTR-STRING
snprintf-S: F_MACRO-STRING
snprintf_P-s: NORMAL-STRING
snprintf_P-S: PSTR-STRING
snprintf_P-S: F_MACRO-STRING
Invalid pointers:
snprintf-s: 
snprintf-s: ⸮⸮⸮
snprintf-S: F_MACRO-STRING
snprintf_P-s: 
snprintf_P-s: 
snprintf_P-S: F_MACRO-STRING

NOTE: The two appearances of "F_MACRO-STRING" under "Invalid pointers:" were RAM pointers to "NORMAL-STRING". Looks like the address of "NORMAL-STRING" in RAM just happened to match the address of "F_MACRO-STRING" in PROGMEM.

The compiler warnings are:

/Users/john/Documents/Arduino/sketch_aug11a/sketch_aug11a.ino: In function 'void setup()':
/Users/john/Documents/Arduino/sketch_aug11a/sketch_aug11a.ino:13:59: warning: format '%S' expects argument of type 'wchar_t*', but argument 4 has type 'const char*' [-Wformat=]
   snprintf(buf, 255, "snprintf-S: %S", PSTR("PSTR-STRING"));
                                                           ^
/Users/john/Documents/Arduino/sketch_aug11a/sketch_aug11a.ino:16:74: warning: format '%S' expects argument of type 'wchar_t*', but argument 4 has type 'const char*' [-Wformat=]
   snprintf(buf, 255, "snprintf-S: %S", (const char *) F("F_MACRO-STRING"));
                                                                          ^
/Users/john/Documents/Arduino/sketch_aug11a/sketch_aug11a.ino:37:55: warning: format '%S' expects argument of type 'wchar_t*', but argument 4 has type 'const char*' [-Wformat=]
   snprintf(buf, 255, "snprintf-S: %S", "NORMAL-STRING");
                                                       ^
Sketch uses 3980 bytes (12%) of program storage space. Maximum is 32256 bytes.
Global variables use 522 bytes (25%) of dynamic memory, leaving 1526 bytes for local variables. Maximum is 2048 bytes.

Perhaps the "-Wformat" argument to the compiler could be used to suppress the spurious warnings.

1 Like

@WPD64, not only the 'const' is in Flash memory, but you should also not use the "%S", just the normal "%s".

The AVR family microcontrollers, such as the Arduino Uno, have a simple and short instruction set. They can not execute code from RAM and they can not have a pointer to data in Flash memory. That is why those special functions, such as 'pgm_read_byte()' are needed to be able to get that data.

The SAMD family processors are more advanced processors, they can use data in Flash.

Make a small test sketch, and try with 'const' and 'PROGMEM' and see what the compiler says about dynamic memory (RAM) usage. Turn on 'verbose' output on 'Compilation' in the preferences.

const char pText[] PROGMEM = 
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789"
  "0123456789012345678901234567890123456789012345678901234567890123456789";

void setup()
{
  Serial.begin( 9600);
  Serial.println( pText);
}

void loop() {}

Arduino Uno:
char pText[] : 888 bytes
const char pText[] : 888
const char pText[] PROGMEM : 188 (data is in Flash memory)

Arduino MKR Wifi 1010:
char pText[] : 2908
const char pText[] : 2204 (data is in Flash memory)
const char pText[] PROGMEM : 2204

The 'PROGMEM' does not give an error with the MKR board, because they made the source code compatible, but it is not needed.

@johnwasser Thank you !
I think I know what I did wrong, when one of the invalid ones is in the sketch, that messes up the ram so bad, that the valid ones will crash.

You missed a few. This one is valid as well:

const char format[] PROGMEM = "snprintf_P-PROGMEM-S-PROGMEM: %S";
const char hello[] PROGMEM = "Hello World";
...
  snprintf_P(buf, 255, format, hello);

@WPD64 this is for the Arduino Uno, not for your board.

I did and can confirm that for const char* and the PROGMEM option will consume FLASH instead of RAM.

But I do not understand how these incompatible pointers are resolved at runtime.
Unless the SAMD-platform employs virtual memory or some other transparent memory-management.

Just to confirm that I understand correctly:

On the SAMD-platform:

  • There's no need at all for PROGMEN or F() to reduce RAM consumption.
  • All const char* will reside in FLASH.
  • snprintf and the like will be fine with (transparently) FLASH-based strings.

That's up to the compiler to decide where to store the data. the C++ specification does not say so

The reason for this behavior ist the complete different architectures of AVR and SAMD processors. The AVR processors have a strict harvard architecture. That means, that programm memory and data memory are completly apart from another and they both have their own adress space, starting from 0. Which type of memory is addressed does not depend from the adress but it depends only on the machine code used. There are only very few machine codes to fetch data from program memory. To create the correct machine code, the compiler must not only know an address, it must also know wether this address is located im program or data memory.

In contrast the SAMD (=arm) processors only have one contigous address space, and part of it is flash and another part is ram. Every machine code can access ram or flash, it is distinguished only by the address ( of course, code that writes to memory will fail in the address range of flash memory ). Therefore the compiler can address (constant) strings in flash in the same manner as in ram.

Duh - C/C++ has no concept of these things :slight_smile:

They are incompatible on the avr processors because RAM, PROGMEM, and EEPROM have separate address spaces. On a processor with only one address space, all pointers are compatible.

Got it.
But why is this distinction even made and why is the concrete location of objects left unclear to the developer while at the same time imposing a strict limit on their respective sizes??

Got it.