Are multiple identical F() strings supposed to reduce to just one in flash?

Projects Programming
1

I thought (many months ago) that I read somewhere (here?) that the same F() string, used multiple times, would be reduced to just ONE instance in flash.
If I'm wrong, then let me know and read no further. Otherwise ...

It doesn't (or at least my following example doesn't for me).
Compiled under Ubuntu 20.04, for Arduino Nano (ATmega328P), using Arduino IDE, a version based on 1.8.20:

#define NUM2STR(N) NUM2STR_(N)
#define NUM2STR_(N) #N
#define BAUDRATE 115200
#define SETTLE_TIME 1000

void setup() {
  // put your setup code here, to run once:
  delay(SETTLE_TIME); // let things settle
  Serial.begin(BAUDRATE); // Serial defined in .../cores/arduino/HardwareSerial.h
  Serial.println(F("\n" __FILE__));
  Serial.println(F(__DATE__ " @" __TIME__  " with BAUDRATE=" NUM2STR(BAUDRATE)));
  Serial.println(F("String#1"));
  Serial.println(F("String#1"));
}

void loop() {
  // put your main code here, to run repeatedly:
}

strings found (ignore $pTAS path) via

find /tmp/arduino_build* \( -iname "*.bin" \)  -type f -exec echo "# {}" >> $pTAS/StringsInBins.txt \; -exec sh -c 'echo "# " $(basename {})' >> $pTAS/StringsInBins.txt \; -exec strings -n3 -tx {} >> $pTAS/StringsInBins.txt \;

gives

## This StringsInBins.txt created via DumpTmpArduinoElfFiles.sh
# /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.with_bootloader.bin
#  DoDuplicateFStringsDisappearFromFlash.ino.with_bootloader.bin
     68 String#1
     71 String#1
     d6 %/0
    23e f#A
    34e -7X
   7f25 $N__O

compilation output:

/home/xxxx/git/Arduino/build/linux/work/arduino-builder -dump-prefs -logger=machine -hardware /home/xxxx/git/Arduino/build/linux/work/hardware -hardware /home/xxxx/.arduino15/packages -tools /home/xxxx/git/Arduino/build/linux/work/tools-builder -tools /home/xxxx/git/Arduino/build/linux/work/hardware/tools/avr -tools /home/xxxx/.arduino15/packages -built-in-libraries /home/xxxx/git/Arduino/build/linux/work/libraries -libraries /home/xxxx/git/BrewArduino/libraries -fqbn=arduino:avr:nano:cpu=atmega328 -vid-pid=1A86_7523 -ide-version=10815 -build-path /tmp/arduino_build_417773 -warnings=default -build-cache /tmp/arduino_cache_830355 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.arduinoOTA.path=/home/xxxx/.arduino15/packages/arduino/tools/arduinoOTA/1.3.0 -prefs=runtime.tools.arduinoOTA-1.3.0.path=/home/xxxx/.arduino15/packages/arduino/tools/arduinoOTA/1.3.0 -prefs=runtime.tools.avr-gcc.path=/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7 -prefs=runtime.tools.avr-gcc-7.3.0-atmel3.6.1-arduino7.path=/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7 -prefs=runtime.tools.avrdude.path=/home/xxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17 -prefs=runtime.tools.avrdude-6.3.0-arduino17.path=/home/xxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17 -verbose /home/xxxx/git/BrewArduino/UnexpectedBehaviour/DoDuplicateFStringsDisappearFromFlash/DoDuplicateFStringsDisappearFromFlash.ino
/home/xxxx/git/Arduino/build/linux/work/arduino-builder -compile -logger=machine -hardware /home/xxxx/git/Arduino/build/linux/work/hardware -hardware /home/xxxx/.arduino15/packages -tools /home/xxxx/git/Arduino/build/linux/work/tools-builder -tools /home/xxxx/git/Arduino/build/linux/work/hardware/tools/avr -tools /home/xxxx/.arduino15/packages -built-in-libraries /home/xxxx/git/Arduino/build/linux/work/libraries -libraries /home/xxxx/git/BrewArduino/libraries -fqbn=arduino:avr:nano:cpu=atmega328 -vid-pid=1A86_7523 -ide-version=10815 -build-path /tmp/arduino_build_417773 -warnings=default -build-cache /tmp/arduino_cache_830355 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.arduinoOTA.path=/home/xxxx/.arduino15/packages/arduino/tools/arduinoOTA/1.3.0 -prefs=runtime.tools.arduinoOTA-1.3.0.path=/home/xxxx/.arduino15/packages/arduino/tools/arduinoOTA/1.3.0 -prefs=runtime.tools.avr-gcc.path=/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7 -prefs=runtime.tools.avr-gcc-7.3.0-atmel3.6.1-arduino7.path=/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7 -prefs=runtime.tools.avrdude.path=/home/xxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17 -prefs=runtime.tools.avrdude-6.3.0-arduino17.path=/home/xxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17 -verbose /home/xxxx/git/BrewArduino/UnexpectedBehaviour/DoDuplicateFStringsDisappearFromFlash/DoDuplicateFStringsDisappearFromFlash.ino
Using board 'nano' from platform in folder: /home/xxxx/.arduino15/packages/arduino/hardware/avr/1.8.6
Using core 'arduino' from platform in folder: /home/xxxx/.arduino15/packages/arduino/hardware/avr/1.8.6
Detecting libraries used...
/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/avr-g++ -c -g -Os -w -std=gnu++11 -Werror -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10815 -DARDUINO_AVR_NANO -DARDUINO_ARCH_AVR -I/home/xxxx/.arduino15/packages/arduino/hardware/avr/1.8.6/cores/arduino -I/home/xxxx/.arduino15/packages/arduino/hardware/avr/1.8.6/variants/eightanaloginputs /tmp/arduino_build_417773/sketch/DoDuplicateFStringsDisappearFromFlash.ino.cpp -o /dev/null
Generating function prototypes...
/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/avr-g++ -c -g -Os -w -std=gnu++11 -Werror -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10815 -DARDUINO_AVR_NANO -DARDUINO_ARCH_AVR -I/home/xxxx/.arduino15/packages/arduino/hardware/avr/1.8.6/cores/arduino -I/home/xxxx/.arduino15/packages/arduino/hardware/avr/1.8.6/variants/eightanaloginputs /tmp/arduino_build_417773/sketch/DoDuplicateFStringsDisappearFromFlash.ino.cpp -o /tmp/arduino_build_417773/preproc/ctags_target_for_gcc_minus_e.cpp
/home/xxxx/git/Arduino/build/linux/work/tools-builder/ctags/5.8-arduino11/ctags -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives /tmp/arduino_build_417773/preproc/ctags_target_for_gcc_minus_e.cpp
Compiling sketch...
/home/xxxx/git/BrewArduino/libraries/BrewmanzUtes/TimestampNow.sh /home/xxxx/git/BrewArduino/UnexpectedBehaviour/DoDuplicateFStringsDisappearFromFlash /tmp/arduino_build_417773
building TimestampNow.h ...  strDate=<2023-07-28 12:19:37.392383589> ...  ... done.
/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/avr-g++ -c -g -Os -std=gnu++11 -Werror -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -MMD -flto -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10815 -DARDUINO_AVR_NANO -DARDUINO_ARCH_AVR -DBL_LFUSE=0xFF -DBL_HFUSE=0xDA -DBL_EFUSE=0xFD "-DPROJECT_NAME=\"DoDuplicateFStringsDisappearFromFlash.ino\"" -I/home/xxxx/.arduino15/packages/arduino/hardware/avr/1.8.6/cores/arduino -I/home/xxxx/.arduino15/packages/arduino/hardware/avr/1.8.6/variants/eightanaloginputs /tmp/arduino_build_417773/sketch/DoDuplicateFStringsDisappearFromFlash.ino.cpp -o /tmp/arduino_build_417773/sketch/DoDuplicateFStringsDisappearFromFlash.ino.cpp.o
Compiling libraries...
Compiling core...
Using precompiled core: /tmp/arduino_cache_830355/core/core_arduino_avr_nano_cpu_atmega328_6ae4461008a515c818fbdb95902010da.a
Linking everything together...
/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/avr-gcc -Os -g -flto -fuse-linker-plugin -Wl,--gc-sections -mmcu=atmega328p -o /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.elf /tmp/arduino_build_417773/sketch/DoDuplicateFStringsDisappearFromFlash.ino.cpp.o /tmp/arduino_build_417773/../arduino_cache_830355/core/core_arduino_avr_nano_cpu_atmega328_6ae4461008a515c818fbdb95902010da.a -L/tmp/arduino_build_417773 -lm
/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/avr-objcopy -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.elf /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.eep
/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/avr-objcopy -O ihex -R .eeprom /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.elf /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.hex
/home/xxxx/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/avr-size -A /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.elf
Sketch uses 1884 bytes (6%) of program storage space. Maximum is 30720 bytes.
Global variables use 188 bytes (9%) of dynamic memory, leaving 1860 bytes for local variables. Maximum is 2048 bytes.
/home/xxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17/bin/avrdude -C/home/xxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17/etc/avrdude.conf -v -patmega328p -carduino -P/dev/ttyUSB0 -b115200 -D -Uflash:w:/tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.hex:i 

avrdude: Version 6.3-20190619
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "/home/xxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17/etc/avrdude.conf"
         User configuration file is "/home/xxxx/.avrduderc"
         User configuration file does not exist or is not a regular file, skipping

         Using Port                    : /dev/ttyUSB0
         Using Programmer              : arduino
         Overriding Baud Rate          : 115200
         AVR Part                      : ATmega328P
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PC2
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff
           flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff
           lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

         Programmer Type : Arduino
         Description     : Arduino
         Hardware Version: 3
         Firmware Version: 4.4
         Vtarget         : 0.3 V
         Varef           : 0.3 V
         Oscillator      : 28.800 kHz
         SCK period      : 3.3 us

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.00s

avrdude: Device signature = 0x1e950f (probably m328p)
avrdude: reading input file "/tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.hex"
avrdude: writing flash (1884 bytes):

Writing | ################################################## | 100% 0.31s

avrdude: 1884 bytes of flash written
avrdude: verifying flash memory against /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.hex:
avrdude: load data flash data from input file /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.hex:
avrdude: input file /tmp/arduino_build_417773/DoDuplicateFStringsDisappearFromFlash.ino.hex contains 1884 bytes
avrdude: reading on-chip flash data:

Reading | ################################################## | 100% 0.24s

avrdude: verifying ...
avrdude: 1884 bytes of flash verified

avrdude done.  Thank you.
2

You're right. It doesn't perform that optimization. It might do that for const data, which would normally be in flash, but isn't on the AVR because the Harvard Architecture makes it difficult.

3

But it could be placed in a function (and called as needed).

1 Like
4

Heh heh. There is almost always some way...

5

You can store the text in a char array in PROGMEM, then reference that when printing instead of using the F() macro. I usually define a macro to save having to cast the char* to FlashStringHelper* each time.

#define NUM2STR(N) NUM2STR_(N)
#define NUM2STR_(N) #N
#define BAUDRATE 115200
#define SETTLE_TIME 1000

#define FF(x) ((__FlashStringHelper*)x)
const char stringNo1[] PROGMEM = "String#1";


void setup() {
  // put your setup code here, to run once:
  delay(SETTLE_TIME); // let things settle
  Serial.begin(BAUDRATE); // Serial defined in .../cores/arduino/HardwareSerial.h
  Serial.println(F("\n" __FILE__));
  Serial.println(F(__DATE__ " @" __TIME__  " with BAUDRATE=" NUM2STR(BAUDRATE)));
  Serial.println(FF(stringNo1));
  Serial.println(FF(stringNo1));
}

void loop() {
  // put your main code here, to run repeatedly:
}
2 Likes
6

Thanks for this example. Had not seen that before. Out of curiosity I wanted to know if the separate function, suggested by @runaway_pancake, or the FlashStringHelper would compile to the smallest size.

#define FF(x) ((__FlashStringHelper*)x)
const char stringNo1[] PROGMEM = "String#1";

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200); 
  Serial.println(FF(stringNo1));
  Serial.println(FF(stringNo1));
  Serial.println(FF(stringNo1));
}

void loop() {
  // put your main code here, to run repeatedly:
}

Compiles to (Atmega168 with Minicore):

Sketch uses 1756 bytes (11%) of program storage space. Maximum is 15872 bytes.
Global variables use 188 bytes (18%) of dynamic memory, leaving 836 bytes for local variables. Maximum is 1024 bytes.

void setup() {
  Serial.begin(115200);
  printstring();
  printstring();
  printstring();
}

void loop() {
  // put your main code here, to run repeatedly:
}

void printstring() {
  Serial.println(F("String#1"));
}

Compiles to (Atmega168 with Minicore):

Sketch uses 1744 bytes (10%) of program storage space. Maximum is 15872 bytes.
Global variables use 188 bytes (18%) of dynamic memory, leaving 836 bytes for local variables. Maximum is 1024 bytes.
7

It depends, if it would be inline then the size will be the same. Noinline not, but maybe after 50th time used. Try it.

8

Try compiling with Tools > CompilerLTO set to "LTO Enabled".

9

Bingo! I thought I had that on by default, but apparently not. No difference then, both compile to:

Sketch uses 1450 bytes (9%) of program storage space. Maximum is 15872 bytes.
Global variables use 188 bytes (18%) of dynamic memory, leaving 836 bytes for local variables. Maximum is 1024 bytes.
10

Thank you to all the people who replied. My delay in responding has been caused with varying reasons, such as an overseas vacation, and much head scratching in trying to achieve what I wanted.
I have a macro, SERIAL_LINE(), which prints the source file name (deliberately with no folder info) and line number; which I use to prefix various messages scattered around my apps with "MySourceFile.xyz(123): ". I then discovered that the "MySourceFile.xyz" string was created multiple times in Flash, once for each SERIAL_LINE() call; Oops.
I have now achieved a single Flash string for multiple occurences of SERIAL_LINE() in a source file, although I admit it's a bit ... ugly.
If anyone has any suggestions on improving it ... go ahead. I was mainly thwarted by limitations of C/C++ macros.
If anyone wants to use my code, or use as a basis for their own requirements ... go ahead - at their own risk, of course.
I have uploaded a Proof of Concept Sketch here for anyone who's interested in how I achieved it.
SerialLinePOC.ino (2.8 KB)
SerialLineUte.cpp (714 Bytes)
SerialLineUte.h (2.1 KB)

11

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.