F() macro garbled c_string

I seem to be having some trouble retrieving char* strings from flash memory.

It is most likely a result of how I am using them.. I have a function to wrap my calls to Serial.print to provide some extra functionality. To make life easier, I have a wrapper to that function that casts the pointer received from the F() macro to a const char * to work with regular c-style strings:

char* getTabs(const int numTabs){
  char* tabs = new char[numTabs + 1];

  for(int i = 0; i < numTabs; i++){
    tabs[i] = "\t";
  }
  tabs[numTabs] = "\0";
  
  return tabs;
}

void writeSerial(bool newline, const char* line){
  if(!SERIAL_LOGS_ENABLED){
    return;
  }
  char* tabs = getTabs(CUR_TAB_LEVEL);
  
  Serial.print(tabs);
  if(newline){
    Serial.println(line);
  } else {
    Serial.print(line);
  }
  delete[] tabs;
}

void writeSerial(const char* line){
  writeSerial(false, line);
}

void writeSerialLine(const char* line){
  writeSerial(true, line);
}

void writeSerial(const __FlashStringHelper * ifsh){
  const char* line = (const char PROGMEM *)ifsh;
  writeSerial(false, line);
}

void writeSerialLine(const __FlashStringHelper * ifsh){
  const char* line = (const char PROGMEM *)ifsh;
  writeSerial(true, line);
}

// usage:

writeSerialLine(F("\n\n\nBegin Setup..."));

Am I casting properly? What could be causing this garbled mess?

Could you give a small sketch that we can try ?

This code is from a much larger sketch spread out over a few files, but I think I have included all relevant code for my issue... I could try to put it in a 'proper' arduino sketch format but figured my problem might be known enough by those that know more than I about how accessing the flash memory works.

My code is on GitHub though: GitHub - GregJohnStewart/WalkingStickPrompter

Direct links:

What happens if you print "Hello" ?

Set the Serial Monitor to 9600 baud and do this:

void setup()
{
  Serial.begin( 9600);
  Serial.println( "Hello");
  ..
}

I have verified Serial is acting properly, it works when I pass regular c-strings directly and through my helpers, as well as when I use the F() macro directly with Serial.println()

You can’t do it like this, your function needs to know if the pointer is for data in RAM or in flash memory so that the correct print method is called. Messing (badly) around with the type is not going to magically move the data into RAM.

You could have two writeSerial() functions, one with a const char * and one with a const __FlashStringHelper * and let the compiler call the right one, which will call the right print method

Also don’t mess around with dynamic memory for your tabs… just send the right amount of tabs with a small loop

for (byte i=0; i< CUR_TAB_LEVEL; i++) Serial.write('\t');

Okay, that is what I was assuming I might need to do, making a second set of functions. I had assumed that the F() macro return type of const __FlashStringHelper * was already in RAM from affect of the macro, and was just a fancy typed pointer.

On the tabs thing I was attempting to avoid all the separate print statements but you are probably right that it is not worth the memory footprint.

Thanks for the help!

The whole idea of the F() macro is to store the string in flash. Because the way an AVR works, flash memory and SRAM are not accessed the same way in a linear way, so what you were doing was taking the memory address in flash and telling the compiler “trust me, it’s a ram address” and then trying to print what is in SRAM at that address ➜ of course that was garbage

I got the idea why you were building the buffer but it’s indeed more costly (and could fail if memory runs low) to allocate the space in the heap, fill it up and then returning the memory to the heap and printing a string is the same as writing each byte individually anyway to the output buffer.

See ArduinoCore-avr/Print.cpp at 24e6edd475c287cdafee0a4db2eb98927ce3cf58 · arduino/ArduinoCore-avr · GitHub


size_t Print::write(const uint8_t *buffer, size_t size)
{
  size_t n = 0;
  while (size--) {
    if (write(*buffer++)) n++;
    else break;
  }
  return n;
}

Is this with a ATmega chip ?
When the code is running, a pointer to data is just a pointer to SRAM. There is no runtime information that the pointer is pointing to Flash memory.
During compilation, the compiler knows which data is in SRAM and which data is in Flash. So the compiler should be told how the route of the data is.

You have to gently tell the compiler to pass on data in Flash to the "Print::print(const __FlashStringHelper *)" function and data in ram to the "Print::print(const char[])" function. They are two different paths and the paths should be separated all the time.

The functions are in the file Print.cpp

Ah, yeah makes sense. Thanks for the clarification!

I also woudn't ask if a quick Google didn't turn up vague and unhelpful, but any tips on getting a const __FlashStringHelper * into a const char*? I am using flash strings elsewhere in an object (and obviously it is garbled as I am casting pointers there)

A ATmega chip can not read data from Flash, it just can't.
Well, with some extra special instructions it can, so it has to use special functions (such as pgm_read_byte) that use those special instructions..

If the compiler knows that data is in Flash and then you strip that information, then the compiler does no longer know it and the data is lost.
You have to use those special functions, copy the data to SRAM, and then you can cast it to a const char *.

Did you read the PROGMEM page ? https://www.arduino.cc/reference/en/language/variables/utilities/progmem/

I'm using an Adafruit Feather 328p, which is a clone of the 328p hardware

Looking at that page I don't see anything quite what I'm looking for, as they are not using the F() macro.

I could probably just work my own thing out by copying the logic from Print.cpp to somehow iterate through the bytes and create a c-str but would rather a simpler way

By the way, as a side note...
In relatively new AVRs such as the ATmega4809 used in the Arduino Nano Every, FLASH ROMs are also contiguous in the memory map and can be accessed with LD/ST instructions.
It doesn't require the F() macro and the compiler places the constants in ROM appropriately.
You can also use the F() macro on such boards for past compatibility, but this will result in poor performance as it will be modified to use LPM/SPM instructions.

Yes, that is a normal way. There are also functions that can copy a number of bytes, such as strcpy_P() and memcpy_P().

Do you understand that there is no pointer to Flash ? You can tell the compiler to treat the two bytes of a pointer in a special way and route that pointer towards pgm_read_... functions.

Do you know when things get really messy ? When a Arduino Mega is used with PROGMEM data of more than 64kbyte. There is still no pointer to Flash, and the compiler can no longer be told to use a pointer of two bytes in a special way.

@chrisknightley Nice, I didn't know that. The sooner we can get rid of PROGMEM, the better.

The whole point of F() is not to eat up SRAM, if you bring the whole thing again in SRAM by making a cString before printing then you loose some of that gain and need to worry about the buffer length etc..

Just duplicate your (improved) function

void writeSerial(bool newline, const char* line){
  if (!SERIAL_LOGS_ENABLED) return;
  for (byte i=0; i< CUR_TAB_LEVEL; i++) Serial.write('\t');
  Serial.print(line);
  if(newline) Serial.println();
}

void writeSerial(bool newline, const __FlashStringHelper * line){
  if (!SERIAL_LOGS_ENABLED) return;
  for (byte i=0; i< CUR_TAB_LEVEL; i++) Serial.write('\t');
  Serial.print(line);
  if(newline) Serial.println();
}

And don’t mess with the pointer type in your functions

void writeSerial(const char* line) { writeSerial(false, line);}
void writeSerialLine(const char* line) {writeSerial(true, line);}
void writeSerial(const __FlashStringHelper * line) {writeSerial(false, line);}
void writeSerialLine(const __FlashStringHelper * line) {writeSerial(true, line);}

And the compiler will do the right thing

If you were using macros that could be even more straightforward (at the cost of more code generated)

1 Like

Yeah, that's why the ATmega4809 has an ROM of 48KB. (Looks a little odded)
Since the address space is 16-bits, it can only be used up to 0xFFFF.
The ROM is allocated from 0x4000 and the available address amount is 0xC000, Yes it's 48KB.

In addition, AVRDA and AVRDB series that can be used as Arduino with DxCore have model with 128KB ROM.
However, they have a slightly nasty address map where only part of the ROM is accessible via LD/ST instructions.

If we all hop on over to the SAMD branch, then the problem is solved :nerd_face:

Yeah, they are excellent. But I even love 8-bit uC ! :blush:

this is how you could approach it with macros:

#if SERIAL_LOGS_ENABLED
#define L_print(...)    {printTabs(); Serial.print(__VA_ARGS__);}
#define L_write(...)    {printTabs(); Serial.print(__VA_ARGS__);}
#define L_println(...)  {printTabs(); Serial.println(__VA_ARGS__);}
#else
#define L_print(...)
#define L_write(...)
#define L_println(...)
#endif

void printTabs() {
  for (byte i=0; i< CUR_TAB_LEVEL; i++) Serial.write('\t');
}

and in you code you just call the L_xxx macros and that will generate the right code (or no code if SERIAL_LOGS_ENABLED is 0 (assuming it’s a #define with a value of 0 or 1)

The benefit is that you can call L_print(variable, HEX) and you will log the value of the variable in hexadecimal or call L_print(float_variable, 5) and log the value with 5 decimals

The negative effect is that a bit more code is generated as you have the call to printTabs() that is added and of course be careful about side effects in parameters for example a call L_print(variable++) will be fine if logs are enabled but the increment won’t happen if logs are disabled since no code will be generated

@J-M-L Yes that is what I will be doing for the Serial output, but I am also using c-strings elsewhere to output text to a TFT screen. It would be nice to be able to get the whole string rather than iterate over bytes.

To be more specific, I am using an Object to help make a menu system, and this object needs to hold either strings I give it (ideally from Flash) or strings from an SD card (from a directory listing, for selecting a file). Unfortunately this would mean copying an entire class just to have one for the flashStringHelpers, which is not acceptible in terms of program storage.

Is there really no predefined helper to get a c-string out of flash? Only the capability to get individual bytes at a time?