Write the same string to multiple outputs in the most Prog Mem efficient way

Hi

I would like to write the same string to multiple outputs in the most PROGMEM efficient way.

Ii I want to do:

  Serial.println(F("initLOG() output.txt Open"));
  Serial1.println(F("initLOG() output.txt Open"));
  logFile.println(F("initLOG() output.txt Open"));

but I assume the code will then have 3 occurrences of the same text in.

In a small test this didnt seem to help me, but perhaps it didnt show a positive result as was only tested against the 1 string:

  const char msg1[] PROGMEM = "initLOG() output.txt Open";
  Serial.println((__FlashStringHelper*)pgm_read_word(msg1));
  Serial1.println((__FlashStringHelper*)pgm_read_word(msg1));
  logFile.println((__FlashStringHelper*)pgm_read_word(msg1));

Thanks very much
Kevin

In a small test this didnt seem to help me

Can you clarify?

You could use:

  const char msg1[] PROGMEM = "initLOG() output.txt Open";
  Serial.println((__FlashStringHelper*)msg1);
  Serial1.println((__FlashStringHelper*)msg1);
  logFile.println((__FlashStringHelper*)msg1);

And lookup the definition of pgm_read_word.

The compiler is smart enough to see that the same string literal is referenced more than once, and will only store one copy of it.

J-M-L:
Can you clarify?

Sorry, yes, I mean: the compiled size of the code size of the two examples came out at exactly the same size, and the second example used more SRAM than the first.

1st:

Sketch uses 6956 bytes (24%) of program storage space. Maximum is 28672 bytes.
Global variables use 876 bytes (34%) of dynamic memory, leaving 1684 bytes for local variables. Maximum is 2560 bytes.

2nd:

Sketch uses 6956 bytes (24%) of program storage space. Maximum is 28672 bytes.
Global variables use 902 bytes (35%) of dynamic memory, leaving 1658 bytes for local variables. Maximum is 2560 bytes.

Thanks very much
Kevin

PaulS:
The compiler is smart enough to see that the same string literal is referenced more than once, and will only store one copy of it.

Ahh, great, thanks, so other than perhaps poor coding style (repeating the same text 3 times in my code), my initial proposed code is the most efficient way?

Thanks very much
Kev

On IDE 1.6.12 (edit: exactly the same result on 1.8.1)

#define ONCE
#ifdef ONCE
const char msg1[] PROGMEM = "initLOG() output.txt Open";
#endif
void setup() {
  Serial.begin(250000);
#ifndef ONCE
  Serial.println(F("initLOG() output.txt Open"));
  Serial1.println(F("initLOG() output.txt Open"));
  Serial2.println(F("initLOG() output.txt Open"));
#else
  Serial.println((__FlashStringHelper*)msg1);
  Serial1.println((__FlashStringHelper*)msg1);
  Serial2.println((__FlashStringHelper*)msg1);
#endif
}

with defined ONCE

2.376 Bytes

without

2.442 Bytes

Or with a different approach to show that the constants do not end in one place

unsigned int lastAdr;

void setup() {
  Serial.begin(250000);
  printWithAdr(F("initLOG() output.txt Open"));
  printWithAdr(F("initLOG() output.txt Open"));
  printWithAdr(F("initLOG() output.txt Open"));
}

void loop() {}

void printWithAdr(const __FlashStringHelper* what) {
  unsigned int adr = (unsigned int) what;
  Serial.print(F("0x"));
  if (adr < 0x1000) {
    Serial.write('0');
    if (adr < 0x100) {
      Serial.write('0');
      if (adr < 0x10) {
        Serial.write('0');
      }
    }
  }
  Serial.print(adr, HEX);
  Serial.print(F(": '"));
  Serial.print(what);
  Serial.print(F("' size 0x"));
  Serial.print(strlen_P((char*)what) + 1, HEX);
  if (lastAdr) {
    Serial.print(F(", distance to last adr 0x"));
    Serial.print(lastAdr - adr, HEX);
  }
  lastAdr = adr;
  Serial.println();
}
0x0143: 'initLOG() output.txt Open' size 0x1A
0x0129: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x1A
0x010F: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x1A

KevWal:

PaulS:
The compiler is smart enough to see that the same string literal is referenced more than once, and will only store one copy of it.

Ahh, great, thanks, so other than perhaps poor coding style (repeating the same text 3 times in my code), my initial proposed code is the most efficient way?

The compiler could be smart enough, but it isn't, despite any wishful thinking (at least with the standard optimization settings).

Reply #2 shows the most memory efficient solution.

Can you try the same test, without wrapping the literal in the F() macro? The compiler SHOULD be able to recognize that the same literal is referenced more than once, and that none of the references involves any possibility if changing the data.

Same result if PSTR() is used instead of F().

unsigned int lastAdr;

void setup() {
  Serial.begin(250000);
  printWithAdr(PSTR("initLOG() output.txt Open"));
  printWithAdr(PSTR("initLOG() output.txt Open"));
  printWithAdr(PSTR("initLOG() output.txt Open"));
}

void loop() {}

void printWithAdr(const char* what) {
  unsigned int adr = (unsigned int) what;
  Serial.print(F("0x"));
  if (adr < 0x1000) {
    Serial.write('0');
    if (adr < 0x100) {
      Serial.write('0');
      if (adr < 0x10) {
        Serial.write('0');
      }
    }
  }
  Serial.print(adr, HEX);
  Serial.print(F(": '"));
  Serial.print((const __FlashStringHelper*)what);
  Serial.print(F("' size 0x"));
  Serial.print(strlen_P((char*)what) + 1, HEX);
  if (lastAdr) {
    Serial.print(F(", distance to last adr 0x"));
    Serial.print(lastAdr - adr, HEX);
  }
  lastAdr = adr;
  Serial.println();
}
0x0143: 'initLOG() output.txt Open' size 0x1A
0x0129: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x1A
0x010F: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x1A

If you put the constant strings to RAM, they are mapped to the same address

unsigned int lastAdr;

void setup() {
  Serial.begin(250000);
  printWithAdr("initLOG() output.txt Open");
  printWithAdr("initLOG() output.txt Open");
  printWithAdr("initLOG() output.txt Open");
}

void loop() {}

void printWithAdr(const char* what) {
  unsigned int adr = (unsigned int) what;
  Serial.print(F("0x"));
  if (adr < 0x1000) {
    Serial.write('0');
    if (adr < 0x100) {
      Serial.write('0');
      if (adr < 0x10) {
        Serial.write('0');
      }
    }
  }
  Serial.print(adr, HEX);
  Serial.print(F(": '"));
  Serial.print(what);
  Serial.print(F("' size 0x"));
  Serial.print(strlen_P((char*)what) + 1, HEX);
  if (lastAdr) {
    Serial.print(F(", distance to last adr 0x"));
    Serial.print(lastAdr - adr, HEX);
  }
  lastAdr = adr;
  Serial.println();
}
0x0210: 'initLOG() output.txt Open' size 0x1A
0x0210: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x0
0x0210: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x0

Thanks very much Whandall, great detective work.

So, to be clear, the right[1] way is:

const char msg1[] PROGMEM = "initLOG() output.txt Open";
  Serial.println((__FlashStringHelper*)msg1);
  Serial1.println((__FlashStringHelper*)msg1);
  logFile.println((__FlashStringHelper*)msg1);

[1] where 'right' is in this case defined as the smallest PROGMEM footprint.

Cheers
Kev

I think so. :slight_smile:

Whandall:
If you put the constant strings to RAM, they are mapped to the same address

THAT is what I was expecting. The literals are duplicated in flash memory, but only one instance gets copied to SRAM. The others point to that one instance.

The folding does not work if the string literals are in PROGMEM.

THAT was the question here.

So, having started to implement this in quite a lot of places in my code, I am finding that whilst the:

 const char m5[] PROGMEM = "setup() Encode SMTP";
SERIAL1_println((__FlashStringHelper*)m5);
SERIAL_println((__FlashStringHelper*)m5);
LOG_println((__FlashStringHelper*)m5);

approach does save me around 5 to 10 bytes of PROGMEM per group of messages, it costs me about 18 to 22 bytes of SRAM for each and every group. So whilst my initial concern was PROGMEM, I am quickly going to run out of SRAM following that approach.

I was surprised to see it consuming SRAM, as I understood FlashStringHelper to enable it to be read directly from PROGMEM.

Infact, removing Const, PROGMEM and the FlashString Helper doesnt seem to change the code size:

  char m6[] = "setup() Ether init";
  SERIAL1_println(m6);
  SERIAL_println(m6);
  LOG_println(m6);

Unfortunataly, then reusing the same variable I need a strcpy, which looks to take up 8 bytes of PROGMEM which is more than the original code did:

  char m6[21] = "setup() Ether init";
  SERIAL1_println(m6);
  SERIAL_println(m6);
  LOG_println(m6);
  if (Ethernet.begin(mac) == 0) {
    strcpy(m6,"setup() Ether failed");
    SERIAL1_println(m6);
    SERIAL_println(m6);
    LOG_println(m6);
    softReset();
  }

Nothing is straight forward, but it might be time to loose the boot loader.....

Cheers
Kev

Please generate a compilable example that shows the described behaviour.
Which Arduino do you use/compile for?

On my 1.8.1 compiles for Mega I get different results.
There is no change in RAM usage, there is a difference in flash usage.

//#define ONCE
#ifdef ONCE
const char msg1[] PROGMEM = "initLOG() output.txt Open";
#endif
void setup() {
  Serial.begin(250000);
#ifndef ONCE
  Serial.println(F("initLOG() output.txt Open"));
  Serial.println(F("initLOG() output.txt Open"));
  Serial.println(F("initLOG() output.txt Open"));
#else
  Serial.println((__FlashStringHelper*)msg1);
  Serial.println((__FlashStringHelper*)msg1);
  Serial.println((__FlashStringHelper*)msg1);
#endif
}
void loop() {}
1828 Bytes 1894 Bytes
 186 Bytes  186 Bytes

Hi

V14 and V15 attached, search for “setup() Read SD” and yuo will see the two sections of code I changed between the two files.

v14 - F("") style - 27668 1889
v15 - F S H style - 27634 1925

I also attached my libraries folder so that it will compile.

I tried your exact code from your post and get the same results as you do (different numbers but same ratio), but in my code I get a different result.

Compiling in 1.8.1 for Leo Eth.

Thanks very much
Kev

SendEmail_014.ino (26.5 KB)

SendEmail_015.ino (26.5 KB)

libraries.zip (853 KB)

Move the definitions of the messages outside any function, or make them static.

This seems to be a problem with local, nonstatic PROGMEM arrays.

#define ONCE
void setup() {
  Serial.begin(250000);
#ifndef ONCE
  Serial.println(F("initLOG() output.txt Open"));
  Serial.println(F("initLOG() output.txt Open"));
  Serial.println(F("initLOG() output.txt Open"));
#else
  const char msg1[] PROGMEM = "initLOG() output.txt Open";
  Serial.println((__FlashStringHelper*)msg1);
  Serial.println((__FlashStringHelper*)msg1);
  Serial.println((__FlashStringHelper*)msg1);
#endif
}
void loop() {}
1880 Bytes
 212 Bytes

with static added

1828 Bytes
 186 Bytes

other versions had

1828 Bytes 1894 Bytes
 186 Bytes  186 Bytes