Storing variable length char array in eeprom?

I'm using the ESP_EEPROM library. I want to declare a structure that will contain several char arrays. Each will be 32 bytes long, for example.
When I try to store a shorter char array, it will throw an error that the types are incompatible: const char[12] incompatible with char[32]. At this times I don't want to use 32 bytes, but at a later point I might use more.

So my question is how to solve this with elegance? I could think of pad the string with some characters and hold another int with the actual length but perhaps there is something nicer? :slight_smile:

plz show code and error

Here is the code:

#include <ESP_EEPROM.h>

struct MyEEPROMStruct 
{
  char    ssid[32];
  char    password[32];
  char    urlPriceQuery[64];
  char    coins[64];
  int       httpsPort;
} eepromVariables;

void setup() 
{
  Serial.begin(115200); // serial monitoring

  // initial values
  eepromVariables.ssid = "ssid ssid ssid";
  eepromVariables.password = "123456786457";
  eepromVariables.urlPriceQuery = "https://api.binance.com/api/v1/ticker/price?symbol="
  eepromVariables.coins = "BTC";

Errors:

wifi_test2:31:24: error: incompatible types in assignment of 'const char [14]' to 'char [32]'
31 | eepromVariables.ssid = "ssid ssid ssid";
wifi_test2:32:28: error: incompatible types in assignment of 'const char [14]' to 'char [32]'
32 | eepromVariables.password = "123456786457";
wifi_test2:33:33: error: incompatible types in assignment of 'const char [52]' to 'char [64]'
33 | eepromVariables.urlPriceQuery = "https://api.binance.com/api/v1/ticker/price?symbol="

  strcpy(eepromVariables.ssid,"ssid ssid ssid");
1 Like

Anyway to make using it safe other than holding the size of the target eeprom array as a variable and compare to it before copy?

strncpy() is safer but still needs the size of the target array

1 Like

use snprintf() to write your strings, it will null terminate, so you can write in larger arrays no problem. Make struct sizes to accomodate the largest string you gonna have and you are sorted.

1 Like

No strncpy() is a bad poor choice, strlcpy is the one to use.
This sketch has examples strlcpy_strlcat.ino

On the other hand (just to be confusing) snprintf() is a good choice.

re strncpy
From Secure Coding in C and C++, 2nd Edition by Robert C. Seacord, Ch5
Because the strncpy() function is not guaranteed to null-terminate the destination string, the programmer must be careful to ensure that the destination string is properly null-terminated without overwriting the last character.
The C Standard strncpy() function is frequently recommended as a “more secure” alternative to strcpy(). However, strncpy() is prone to string termination errors, as detailed shortly under “C11 Annex K Bounds-Checking Interfaces.”

Opinions seem to vary
From https://documentation.help/Cpp/strlcpy.htm

Why strlcpy is not an improvement, but rather a different, and quite possibly worse compromise:

Looks like cods wallop to me
"with strncpy a look at the dump will tell you exactly what the problem is."
Well no, not on Arduino Uno/Meg2650 etc, the sketch just crashes with no indication of where or what.

The entire document is written for computers, not micros, and the author seems to think that core dumps are an acceptable form of error message.

The document also assumes that the user does NOT check that return of strlcpy.
after strlcpy(buf,”assume”,3); nothing whatsoever will show that the copy has failed,

This statement is just false.

There was a clear indication of the failure in the return value, the author just cannot be bothered to check. (see strlcpy_strlcat.ino for just how simple the check is)
It is always a bad idea to ignore error returns, but not surprising from someone who uses core dumps for error checking.

What is so good about strlcpy for Arduino is that if it fails your sketch does not crash you just get empty an string which you can debug. However with strlcpy you can also check the return and flag exactly where the problem occurs.

I am completely ignoring the discussion about speed of execution when working with strings 42Kbyte long, which is just way out of the realm of AVR boards (and many other micros)

So no strncpy is NOT the way to go, strlcpy/strlcat are the safe methods (no crashes later due to missing terminating '\0') even if you don't bother to check the return, which you should, you can still add debug print(), that will be executed, that will be executed so you can narrow down where the problem is.

As I said, opinions vary

That's rich coming from the user who keeps telling us that Microsoft have banned the use of C style strings when Microsoft do not write software for micros

Breaking news !
Not using software properly causes problems

Interesting comments but they do nothing to support the author's contention that strncpy is some how safer/better than strlcpy.

Are you proposing that Arduino programmers should use strncpy and just let the sketch crash on a unterminated result, as the author suggests, instead of using strlcpy and not have their sketch crash and also be able to pinpoint the error as it occurs by a simple test on the return from strlcpy.

And of course Microsoft could change their mind.

Remember when Windows 10 was the 'last version of Windows' ?

I am proposing that programmers use functions properly. Nothing more, nothing less

@UKHeliBob Some example code that illustrates how you think strncpy/strncat should be used properly as an alternative to strlcpy_strlcat.ino would be helpful.

OK. Here are some examples for you to criticize, which no doubt you will

See https://forum.arduino.cc/t/re-storing-variable-length-char-array-in-eeprom/881399 for further discussion of a problem with this code

If they are written incorrectly I will accept the fact but if they are correct I will not accept the possibility that a programmer might make a mistake when using them because that is obvious

The point is that used correctly the functions are safe, not whether they are easy to use, which I accept that they are not, especially strncat()

char dest[10];  //space for 9 characters plus a terminating '\0'
byte maxDestChars = sizeof(dest) - 1;  //max number of destination characters allowing for termination
byte charsRemaining;  //number of free chars in destination for concatenation

void setup()
{
  Serial.begin(115200);
  while (!Serial);
  strncpy(dest, "12345", maxDestChars); //should be plenty of room for these characters
  Serial.println(dest);
  //
  strncpy(dest, "123456789", maxDestChars); //should be just enough room for these characters
  Serial.println(dest);
  //
  strncpy(dest, "1234567890ABCDEF", maxDestChars);  //far too many characters to fit
  Serial.println(dest);
  //
  strncpy(dest, "X", maxDestChars); //start with an X in the destination
  charsRemaining = maxDestChars - strlen(dest); //calculate remaining space in destination
  strncat(dest, "12345", charsRemaining); //plenty of room to add these characters
  Serial.println(dest);
  //
  strncpy(dest, "X", maxDestChars); //start with an X in the destination
  charsRemaining = maxDestChars - strlen(dest); //calculate remaining space in destination
  strncat(dest, "12345678", charsRemaining); //enough room for these characters
  Serial.println(dest);
  //
  strncpy(dest, "X", maxDestChars); //start with an X in the destination
  charsRemaining = maxDestChars - strlen(dest); //calculate remaining space in destination
  strncat(dest, "1234567890", charsRemaining); //too many characters to fit in destination
  Serial.println(dest);
}

void loop()
{
}

Note that I chose to write the code inline for clarity but in real life the mechanics of how it works would probably be put into a function and called as required with parameters thus allowing thorough testing and validation

Thanks @UKHeliBob
If you are ignoring method error returns (and lots of coders ignore error returns) than not much difference between strncpy and strlcpy,

The advantage strlcpy/strlcat have is that you can easily check for truncation errors. That option is missing from strncpy/strncat and that is the first reason I consider strlcpy/strlcat is the better choice.
The second reason is that coders tend to use strncpy/strncat as a pair and then get the strncat method args wrong. They don't do the correct calculation as you have done. strlcat is a simpler choice as it does the calc for you internally.

Here is the strlcpy strlcat version with error checking

char dest[10];  //space for 9 characters plus a terminating '\0'

void setup()
{
  Serial.begin(115200);
  for (int i = 10; i > 0; i--) {
    delay(500);
    Serial.print(i); Serial.print(' ');
  }
  Serial.println("Uno started ");
  if (strlcpy(dest, "12345", sizeof(dest)) >= sizeof(dest)) { //should be plenty of room for these characters
    Serial.println(F("Error dest too small"));
  } else {
    Serial.println(dest);
  }
  //
  if (strlcpy(dest, "123456789", sizeof(dest)) >=  sizeof(dest)) { //should be just enough room for these characters
    Serial.println(F("Error dest too small"));
  } else {
    Serial.println(dest);
  }
  //
  if (strlcpy(dest, "1234567890ABCDEF", sizeof(dest)) >= sizeof(dest)) {  //far too many characters to fit
    Serial.println(F("Error dest too small"));
  } else {
    Serial.println(dest);
  }
  //
  if (strlcpy(dest, "X", sizeof(dest)) >= sizeof(dest)) { //start with an X in the destination
    Serial.println(F("Error dest too small"));
  }
  if (strlcat(dest, "12345", sizeof(dest))  >= sizeof(dest)) { //plenty of room to add these characters
    Serial.println(F("Error dest too small"));
  } else {
    Serial.println(dest);
  }
  //
  if (strlcpy(dest, "X", sizeof(dest))  >= sizeof(dest)) { //start with an X in the destination
    Serial.println(F("Error dest too small"));
  }
  if (strlcat(dest, "12345678", sizeof(dest))  >= sizeof(dest)) { //enough room for these characters
    Serial.println(F("Error dest too small"));
  } else {
    Serial.println(dest);
  }
  //
  if (strlcpy(dest, "X", sizeof(dest))  >= sizeof(dest)) { //start with an X in the destination
    Serial.println(F("Error dest too small"));
  }
  if (strlcat(dest, "1234567890", sizeof(dest)) >= sizeof(dest)) { //too many characters to fit in destination
    Serial.println(F("Error dest too small"));
  } else {
    Serial.println(dest);
  }
}

void loop()
{
}

Your output is (the same as strlcpy/ strlcat without error checking)

12345
123456789
123456789
X12345
X12345678
X12345678

The strlcpy/strlcat output with error checking is

12345
123456789
Error dest too small
X12345
X12345678
Error dest too small

Personally I prefer SafeString, I find the code simpler and the error msgs much better.

#include "SafeString.h"
char dest[10];  //space for 9 characters plus a terminating '\0'

void setup()
{
  Serial.begin(115200);
  for (int i = 10; i > 0; i--) {
    delay(500);
    Serial.print(i); Serial.print(' ');
  }
  Serial.println("Uno started ");
  SafeString::setOutput(Serial);
  
  cSFA(sfDest,dest); // wrap in a safe string
  sfDest = "12345";  //should be plenty of room for these characters
  Serial.println(dest);
  //
  sfDest = "123456789"; //should be just enough room for these characters
  Serial.println(dest);
  //
  sfDest = "1234567890ABCDEF";  //far too many characters to fit
  Serial.println(dest);
  //
  sfDest = "X"; //start with an X in the destination
  sfDest += "12345"; //plenty of room to add these characters
  Serial.println(dest);
  //
  sfDest = "X"; //start with an X in the destination
  sfDest += "12345678"; //enough room for these characters
  Serial.println(dest);
  //
  sfDest = "X"; //start with an X in the destination
  sfDest += "1234567890"; //too many characters to fit in destination
  Serial.println(dest);
}

void loop()
{
}

The output is

12345
123456789
Error: sfDest = "1234567890ABCDEF"
 needs capacity of 16
        sfDest cap:9 len:9 '123456789'

X12345
X12345678
Error: sfDest.concat() needs capacity of 11(i.e. char[12]) for the first 10 chars of the input.
        Input arg was '1234567890'
        sfDest cap:9 len:1 'X'
X

BTDTGTTS when writing the examples, so I know what you mean. Of course, the error of my ways showed up when I tested the code. Then I "corrected" the code but unthinkingly used sizeof() instead of strlen() when calculating the remaining space in the destination. Again, testing showed that there was a problem and I fixed it by turning on my brain and using the correct function

Using the return values from the alternative functions raises the interesting question as to what to do it if the return value indicates that there has been a problem and the same goes for SafeStrings

We should start another whole topic on that. In my professional experience I spend as significant amount of time and code working out what to do with errors. In Java you see lots of code, even in Oracle's libraries, like this

try {
 . . . 
} catch (Exception ex) {
  // no where to send this, oh well keep going
}

My Java GUI Programming Tips and Guidelines - Error Recovery page goes into this.

Error handling is why a added SafeString::Output.
By default is it is a bin bucket, but if you call
SafeString::setOutput(Serial); // or set some other Stream (say WiFi)
Then
SafeString::Output.print(.. );
will output there.

This lets you output error messages from deep within your libraries and globally enable/disable all messages. You can also to turn Output off and on programatically with SafeString::turnOutputOff(); SafeString::setOutput(Serial);

Because SafeString::Output is such a mouth full I usually use
#define debugOut SafeString::Output
Then I can safely use debugOut.print( ) every where in that file.

BUT all this is moot if you don't have anywhere to display the errors (headless micros) In that case I set a 'sensible' default OR an 'invalid' value that will be ignored elsewhere, and continue. Edit --- actually you shouldto do this even if you print an Error msg somewhere.

Edit - SafeString has a per object error flag you can check with sfStr.hasError() and also a global error flag set by any SafeString error SafeString::errorDetected(). So you can still programatically 'detect' errors even if all the error msgs are turned of or there is no where to send them. These flags both clear on each call.

1 Like

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.