Write the same string to multiple outputs in the most Prog Mem efficient way

KevWal · February 28, 2017, 10:05pm

Hi

I would like to write the same string to multiple outputs in the most PROGMEM efficient way.

Ii I want to do:

  Serial.println(F("initLOG() output.txt Open"));
  Serial1.println(F("initLOG() output.txt Open"));
  logFile.println(F("initLOG() output.txt Open"));

but I assume the code will then have 3 occurrences of the same text in.

In a small test this didnt seem to help me, but perhaps it didnt show a positive result as was only tested against the 1 string:

  const char msg1[] PROGMEM = "initLOG() output.txt Open";
  Serial.println((__FlashStringHelper*)pgm_read_word(msg1));
  Serial1.println((__FlashStringHelper*)pgm_read_word(msg1));
  logFile.println((__FlashStringHelper*)pgm_read_word(msg1));

Thanks very much
Kevin

J-M-L · February 28, 2017, 10:14pm

In a small test this didnt seem to help me

Can you clarify?

Whandall · February 28, 2017, 10:21pm

You could use:

  const char msg1[] PROGMEM = "initLOG() output.txt Open";
  Serial.println((__FlashStringHelper*)msg1);
  Serial1.println((__FlashStringHelper*)msg1);
  logFile.println((__FlashStringHelper*)msg1);

And lookup the definition of pgm_read_word.

system · February 28, 2017, 10:46pm

The compiler is smart enough to see that the same string literal is referenced more than once, and will only store one copy of it.

KevWal · February 28, 2017, 10:55pm

J-M-L:
Can you clarify?

Sorry, yes, I mean: the compiled size of the code size of the two examples came out at exactly the same size, and the second example used more SRAM than the first.

1st:

Sketch uses 6956 bytes (24%) of program storage space. Maximum is 28672 bytes.
Global variables use 876 bytes (34%) of dynamic memory, leaving 1684 bytes for local variables. Maximum is 2560 bytes.

2nd:

Sketch uses 6956 bytes (24%) of program storage space. Maximum is 28672 bytes.
Global variables use 902 bytes (35%) of dynamic memory, leaving 1658 bytes for local variables. Maximum is 2560 bytes.

Thanks very much
Kevin

KevWal · February 28, 2017, 11:01pm

PaulS:
The compiler is smart enough to see that the same string literal is referenced more than once, and will only store one copy of it.

Ahh, great, thanks, so other than perhaps poor coding style (repeating the same text 3 times in my code), my initial proposed code is the most efficient way?

Thanks very much
Kev

Whandall · February 28, 2017, 11:05pm

On IDE 1.6.12 (edit: exactly the same result on 1.8.1)

#define ONCE
#ifdef ONCE
const char msg1[] PROGMEM = "initLOG() output.txt Open";
#endif
void setup() {
  Serial.begin(250000);
#ifndef ONCE
  Serial.println(F("initLOG() output.txt Open"));
  Serial1.println(F("initLOG() output.txt Open"));
  Serial2.println(F("initLOG() output.txt Open"));
#else
  Serial.println((__FlashStringHelper*)msg1);
  Serial1.println((__FlashStringHelper*)msg1);
  Serial2.println((__FlashStringHelper*)msg1);
#endif
}

with defined ONCE

2.376 Bytes

without

2.442 Bytes

Whandall · February 28, 2017, 11:44pm

Or with a different approach to show that the constants do not end in one place

unsigned int lastAdr;

void setup() {
  Serial.begin(250000);
  printWithAdr(F("initLOG() output.txt Open"));
  printWithAdr(F("initLOG() output.txt Open"));
  printWithAdr(F("initLOG() output.txt Open"));
}

void loop() {}

void printWithAdr(const __FlashStringHelper* what) {
  unsigned int adr = (unsigned int) what;
  Serial.print(F("0x"));
  if (adr < 0x1000) {
    Serial.write('0');
    if (adr < 0x100) {
      Serial.write('0');
      if (adr < 0x10) {
        Serial.write('0');
      }
    }
  }
  Serial.print(adr, HEX);
  Serial.print(F(": '"));
  Serial.print(what);
  Serial.print(F("' size 0x"));
  Serial.print(strlen_P((char*)what) + 1, HEX);
  if (lastAdr) {
    Serial.print(F(", distance to last adr 0x"));
    Serial.print(lastAdr - adr, HEX);
  }
  lastAdr = adr;
  Serial.println();
}

0x0143: 'initLOG() output.txt Open' size 0x1A
0x0129: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x1A
0x010F: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x1A

Whandall · March 1, 2017, 7:43am

KevWal:

PaulS:
The compiler is smart enough to see that the same string literal is referenced more than once, and will only store one copy of it.

Ahh, great, thanks, so other than perhaps poor coding style (repeating the same text 3 times in my code), my initial proposed code is the most efficient way?

The compiler could be smart enough, but it isn't, despite any wishful thinking (at least with the standard optimization settings).

Reply #2 shows the most memory efficient solution.

system · March 1, 2017, 10:48am

Can you try the same test, without wrapping the literal in the F() macro? The compiler SHOULD be able to recognize that the same literal is referenced more than once, and that none of the references involves any possibility if changing the data.

Whandall · March 1, 2017, 10:54am

Same result if PSTR() is used instead of F().

unsigned int lastAdr;

void setup() {
  Serial.begin(250000);
  printWithAdr(PSTR("initLOG() output.txt Open"));
  printWithAdr(PSTR("initLOG() output.txt Open"));
  printWithAdr(PSTR("initLOG() output.txt Open"));
}

void loop() {}

void printWithAdr(const char* what) {
  unsigned int adr = (unsigned int) what;
  Serial.print(F("0x"));
  if (adr < 0x1000) {
    Serial.write('0');
    if (adr < 0x100) {
      Serial.write('0');
      if (adr < 0x10) {
        Serial.write('0');
      }
    }
  }
  Serial.print(adr, HEX);
  Serial.print(F(": '"));
  Serial.print((const __FlashStringHelper*)what);
  Serial.print(F("' size 0x"));
  Serial.print(strlen_P((char*)what) + 1, HEX);
  if (lastAdr) {
    Serial.print(F(", distance to last adr 0x"));
    Serial.print(lastAdr - adr, HEX);
  }
  lastAdr = adr;
  Serial.println();
}

0x0143: 'initLOG() output.txt Open' size 0x1A
0x0129: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x1A
0x010F: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x1A

Whandall · March 1, 2017, 10:58am

If you put the constant strings to RAM, they are mapped to the same address

unsigned int lastAdr;

void setup() {
  Serial.begin(250000);
  printWithAdr("initLOG() output.txt Open");
  printWithAdr("initLOG() output.txt Open");
  printWithAdr("initLOG() output.txt Open");
}

void loop() {}

void printWithAdr(const char* what) {
  unsigned int adr = (unsigned int) what;
  Serial.print(F("0x"));
  if (adr < 0x1000) {
    Serial.write('0');
    if (adr < 0x100) {
      Serial.write('0');
      if (adr < 0x10) {
        Serial.write('0');
      }
    }
  }
  Serial.print(adr, HEX);
  Serial.print(F(": '"));
  Serial.print(what);
  Serial.print(F("' size 0x"));
  Serial.print(strlen_P((char*)what) + 1, HEX);
  if (lastAdr) {
    Serial.print(F(", distance to last adr 0x"));
    Serial.print(lastAdr - adr, HEX);
  }
  lastAdr = adr;
  Serial.println();
}

0x0210: 'initLOG() output.txt Open' size 0x1A
0x0210: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x0
0x0210: 'initLOG() output.txt Open' size 0x1A, distance to last adr 0x0

KevWal · March 1, 2017, 12:40pm

Thanks very much Whandall, great detective work.

So, to be clear, the right[1] way is:

const char msg1[] PROGMEM = "initLOG() output.txt Open";
  Serial.println((__FlashStringHelper*)msg1);
  Serial1.println((__FlashStringHelper*)msg1);
  logFile.println((__FlashStringHelper*)msg1);

[1] where 'right' is in this case defined as the smallest PROGMEM footprint.

Cheers
Kev

Whandall · March 1, 2017, 12:46pm

I think so.

system · March 1, 2017, 8:08pm

Whandall:
If you put the constant strings to RAM, they are mapped to the same address

THAT is what I was expecting. The literals are duplicated in flash memory, but only one instance gets copied to SRAM. The others point to that one instance.

Whandall · March 1, 2017, 8:12pm

The folding does not work if the string literals are in PROGMEM.

THAT was the question here.

KevWal · March 1, 2017, 10:18pm

So, having started to implement this in quite a lot of places in my code, I am finding that whilst the:

 const char m5[] PROGMEM = "setup() Encode SMTP";
SERIAL1_println((__FlashStringHelper*)m5);
SERIAL_println((__FlashStringHelper*)m5);
LOG_println((__FlashStringHelper*)m5);

approach does save me around 5 to 10 bytes of PROGMEM per group of messages, it costs me about 18 to 22 bytes of SRAM for each and every group. So whilst my initial concern was PROGMEM, I am quickly going to run out of SRAM following that approach.

I was surprised to see it consuming SRAM, as I understood FlashStringHelper to enable it to be read directly from PROGMEM.

Infact, removing Const, PROGMEM and the FlashString Helper doesnt seem to change the code size:

  char m6[] = "setup() Ether init";
  SERIAL1_println(m6);
  SERIAL_println(m6);
  LOG_println(m6);

Unfortunataly, then reusing the same variable I need a strcpy, which looks to take up 8 bytes of PROGMEM which is more than the original code did:

  char m6[21] = "setup() Ether init";
  SERIAL1_println(m6);
  SERIAL_println(m6);
  LOG_println(m6);
  if (Ethernet.begin(mac) == 0) {
    strcpy(m6,"setup() Ether failed");
    SERIAL1_println(m6);
    SERIAL_println(m6);
    LOG_println(m6);
    softReset();
  }

Nothing is straight forward, but it might be time to loose the boot loader.....

Cheers
Kev

Whandall · March 1, 2017, 10:35pm

Please generate a compilable example that shows the described behaviour.
Which Arduino do you use/compile for?

On my 1.8.1 compiles for Mega I get different results.
There is no change in RAM usage, there is a difference in flash usage.

//#define ONCE
#ifdef ONCE
const char msg1[] PROGMEM = "initLOG() output.txt Open";
#endif
void setup() {
  Serial.begin(250000);
#ifndef ONCE
  Serial.println(F("initLOG() output.txt Open"));
  Serial.println(F("initLOG() output.txt Open"));
  Serial.println(F("initLOG() output.txt Open"));
#else
  Serial.println((__FlashStringHelper*)msg1);
  Serial.println((__FlashStringHelper*)msg1);
  Serial.println((__FlashStringHelper*)msg1);
#endif
}
void loop() {}

1828 Bytes 1894 Bytes
 186 Bytes  186 Bytes

KevWal · March 2, 2017, 5:57pm

Hi

V14 and V15 attached, search for "setup() Read SD" and yuo will see the two sections of code I changed between the two files.

v14 - F("") style - 27668 1889
v15 - F S H style - 27634 1925

I also attached my libraries folder so that it will compile.

I tried your exact code from your post and get the same results as you do (different numbers but same ratio), but in my code I get a different result.

Compiling in 1.8.1 for Leo Eth.

Thanks very much
Kev

SendEmail_014.ino (26.5 KB)

SendEmail_015.ino (26.5 KB)

libraries.zip (853 KB)

Whandall · March 2, 2017, 6:18pm

Move the definitions of the messages outside any function, or make them static.

This seems to be a problem with local, nonstatic PROGMEM arrays.

#define ONCE
void setup() {
  Serial.begin(250000);
#ifndef ONCE
  Serial.println(F("initLOG() output.txt Open"));
  Serial.println(F("initLOG() output.txt Open"));
  Serial.println(F("initLOG() output.txt Open"));
#else
  const char msg1[] PROGMEM = "initLOG() output.txt Open";
  Serial.println((__FlashStringHelper*)msg1);
  Serial.println((__FlashStringHelper*)msg1);
  Serial.println((__FlashStringHelper*)msg1);
#endif
}
void loop() {}

1880 Bytes
 212 Bytes

with static added

1828 Bytes
 186 Bytes

other versions had

1828 Bytes 1894 Bytes
 186 Bytes  186 Bytes

Topic		Replies	Views
PROGMEM usage Programming Questions	14	27890	May 5, 2021
Printing from PROGMEM in simplest fashion? Programming Questions	5	1740	May 5, 2021
are strings (defined as arguments for a function) stored in RAM or PGM memory? Programming Questions	4	608	May 5, 2021
PROGMEM and string retrieval Programming Questions	22	9863	May 6, 2021
progmem function Programming Questions	2	1190	May 5, 2021

Write the same string to multiple outputs in the most Prog Mem efficient way

Related topics