String vs Array memory Usage

Projects Programming
1

I am always being advised not to use String class variables and to use char arrays instead.

I did notice earlier when I changed a String variable to a char variable[], there did not seem to be much difference in memory usage.

In any case I did a little experiment and in my sketch changed 2 String class variables to char arrays.
The posted code are the differences between the 2 sketches.

Original code (extract):

...
String cid;
String sysRef;
...
void setup()
  {
...
    cid = Ref1;
    cid += "-";
    cid += Ref2;
...
    sysRef = cid;
    sysRef += "_";
    sysRef += Ref3;
    sysRef += ":";
    sysRef += Ref4;
... 
 
void loop() {
  // put your main code here, to run repeatedly:

}

Sketch uses 26,736 bytes (82%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,600 bytes (78%) of dynamic memory, leaving 448 bytes for local variables. Maximum is 2,048 bytes.

Revised code...

...
char cid[10];
char sysRef[16];
...
void setup()
  {
...
    sprintf(cid, ("%s-%s"), Ref1, Ref2);
...
    sprintf(sysRef, ("%s_%s:%s"), cid, Ref3, Ref4);
...
 
void loop() {
  // put your main code here, to run repeatedly:

}

Sketch uses 28,036 bytes (86%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,616 bytes (78%) of dynamic memory, leaving 432 bytes for local variables. Maximum is 2,048 bytes.

So I see my use of String class variables uses less memory and I wonder why when everyone tells me it should be the opposite.

So the first question is, is there a more efficient way to derive cid and sysRef other than using sprintf?

FWIW it should be noted that the sketch does contain other String class variables.

2

The problem with String isn't the memory usage, it's the fragmentation of the heap...

1 Like
3

FWIW it should be noted that the sketch does contain other String class variables.

So, removing two instances is not going to save any program space.

If all the variables being manipulated by sprintf() are strings, using sprintf() is unnecessary. strcat() will concatenate the strings and formatting information (the _, -, and : ) with less code space and memory requirements.

Where sprintf() is useful is when you need to format an int as a string of hex digits, with leading 0x and exactly 4 digits after the x, for instance. No other function can do that formatting.

If you just want to convert an int to a string, without special formatting, use itoa().

4

As long as one variable uses the String class, a certain amount of overhead will be pulled into your code. The same is true for sprintf(). As a general rule, I avoid both whenever possible. sprintf() is a very powerful and flexible function, but it has high memory requirements if you are only using a small subset of its functionality. The same is true for the String class--it's very easy to use, but it's a memory glutton compared to alternative ways using char arrays. If you want to see the amount of memory that can be saved, dump all references to the String class and replace them with char arrays.

5

PaulS:
If all the variables being manipulated by sprintf() are strings, using sprintf() is unnecessary. strcat() will concatenate the strings and formatting information (the _, -, and : ) with less code space and memory requirements.

Thanks Paul.

I now have the code as follows :

...
char cid[10];
char sysRef[16];
...
void setup()
  {
...
    strcpy(cid, Ref1);
    strcat(cid, "-");
    strcat(cid, Ref2);
...    
    strcpy(sysRef, cid);
    strcat(sysRef, "_");
    strcat(sysRef, Ref3);
    strcat(sysRef, ":");
    strcat(sysRef, Ref4);
...

My memory usage has dropped back down:

Sketch uses 26,718 bytes (82%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,606 bytes (78%) of dynamic memory, leaving 442 bytes for local variables. Maximum is 2,048 bytes.

A further question...
For the above I knew the length of the variables as they are fixed length, so I knew what length to set the arrays.
When working with a variable of unknown length but the max length is known, are there any issues to deal with if the variable is shorter than the char array[] length?

6

The bit you may not have understood, is whether using the String class will use more of the remaining local variable space, of which you have only about 400 bytes.

7

michinyon:
The bit you may not have understood, is whether using the String class will use more of the remaining local variable space, of which you have only about 400 bytes.

michinyon:
The bit you may not have understood, is whether using the String class will use more of the remaining local variable space, of which you have only about 400 bytes.

Following econjack's post #3 I do (now) understand that point and am working towards getting rid of all String class variables, after which hopefully I will see a bigger saving of resources.

8

It may be (I can't tell because you didn't post your code) that the string (or String) operations are unnecessary.
If all you're doing is concatenating strings for transmission, you may find it's just as simple to transmit the partial strings (or Strings) individually as they are.

9

Well I got rid of all the String class variables and ended up with the following memory usage:

Sketch uses 24,878 bytes (77%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,766 bytes (86%) of dynamic memory, leaving 282 bytes for local variables. Maximum is 2,048 bytes.

Going back to where I started out from :

24,878 bytes (77%) vs 26,658 bytes (82%) of program storage space - using 5% less
1,766 bytes (86%) vs 1,694 bytes (82%) of dynamic memory - using 4% more

Probably still not good enough to run on an Uno.
I suppose next step is to use Progmem...

10

Why not post all of your code? There may well be other ways that you could save memory.

11

A further question...
When working with a variable of unknown length but the max length is known, are there any issues to deal with if the variable is shorter than the char array[] length?

No, that is just lost memory space unavailable to any other operation not assigned to that array space.

12

PaulS:
Why not post all of your code? There may well be other ways that you could save memory.

Actually my main objective was to learn how to NOT use the String class of variables - which I can now do; and at the same time I wanted to understand the implications of memory usage.

I am more trying to learn and better understand coding the Arduino than optimising the sketch.

FWIW I started out using an Uno, but considering the hardware used in the sketch now already includes Ethernet shield, SD card, 433MHz receiver and I2C LCD; in addition to which I want to add a Bluetooth module and optional WiFi, I have switched to a Mega as I figure there is Buckley's of it fitting onto an Uno.

13

zoomkat:
No, that is just lost memory space unavailable to any other operation not assigned to that array space.

Thanks zoomkat.

Another follow-on question....
My understanding is the compiler automatically set the null termination in an initialized array.
Taking a declared but uninitialized array, once I have written everything to the array that I want in the array, do I need to explicitly set the null termination?

var[sizeof(var) - 1] = '\0';
14

do I need to explicitly set the null termination?

Yes, but NOT that way. You should be adding the NULL every time you add to the array.

  array[index++] = newData;
  array[index] = '\0';

The NULL is a stop sign. Imagine starting a roadbed (declaring an array), paving part of it (adding data to the array), and then putting up a stop sign. Where do you put the stop sign? At the end of where you are going to pave, as you are doing? Or, at the end of where you have paved, as my snippet does?

15

PaulS:
Yes, but NOT that way. You should be adding the NULL every time you add to the array.

  array[index++] = newData;

array[index] = '\0';




The NULL is a stop sign. Imagine starting a roadbed (declaring an array), paving part of it (adding data to the array), and then putting up a stop sign. Where do you put the stop sign? At the end of where you are going to pave, as you are doing? Or, at the end of where you have paved, as my snippet does?

Ok I totally get what you are saying, but I had to experiment with some arrays before replying.

I believe if I am writing to an array with a "for loop" where the array is bigger than needed to hold all the elements, then your method is ideal.

However, what if I am writing to an array with strcpy() and concatenating with strcat() to form a single string variable AND where the length of each piece of joined data is of known/fixed length and fills the array exactly except for the last position (see var4 in example code), then would my method be appropriate?

Or do you have an alternative suggestion as to how to add a null termination to a "concatenated" array...?

Now before you say I don't need to concatenate an array and should just use the individual variables, I find it easier to formulate a single string variable when posting to long list of variables to an online database.

The good thing about your method is as long as the array is sized too big, all will be well. The downside is you cannot use sizeof() since it won't be correct.
With my method, the sizing needs to be precise (i.e. manual calculation of size), but you can use sizeof().

In the example code var1 and var4 are null terminated. Var2 and var3 are a hypothetical case, but for the sake of the discussion, what would the implications be to code execution if they were used without being null terminated?

Thanks for the input so far :wink:

  char var1[] = "abcdef";
  char var2[7];
  char var3[7];
  char var4[11];
  
void setup() {
  // put your setup code here, to run once:
  char x[] = "9999";
  char y[] = "123456";
  
  Serial.begin(9600);
  while(!Serial);
  
  Serial.println(var1);
  int a = sizeof(var1);
  Serial.println(a); // 7
  
  strcpy(var2, x);
  Serial.println(var2);
  int b = sizeof(var2);
  Serial.println(b); // 7 (5)
  
  strcpy(var3, y);
  Serial.println(var3);
  int c = sizeof(var3);
  Serial.println(c); // 7

  strcpy(var4, x);
  strcat(var4, y);
  Serial.println(var4);
  var4[sizeof(var4) - 1] = '\0';
  // or : var4[10] = '\0';
  int d = sizeof(var4);
  Serial.println(d); // 11
   
}

void loop() {
  // put your main code here, to run repeatedly:

}
16

However, what if I am writing to an array with strcpy() and concatenating with strcat() to form a single string variable AND where the length of each piece of joined data is of known/fixed length and fills the array exactly except for the last position (see var4 in example code), then would my method be appropriate?

No, it wouldn't. strcpy() and strcat() takes strings (not just char arrays) as arguments. The input strings are NULL terminated. The output string is NULL terminated.

The good thing about your method is as long as the array is sized too big, all will be well. The downside is you cannot use sizeof() since it won't be correct.

There is no downside. The size of a string is determined by strlen(), not sizeof().

17

I'm missing something here. The sizeof() operator is used to return the memory space that's been allocated to a variable. It's not going to vary regardless of what you put in the array. If you want to find out how much of an array is actually used, strlen() can provide that information, as seen here:

  char var1[] = "abcdef";
  char var2[7];
  char var3[7];
  char var4[11];
  
void setup() {
  // put your setup code here, to run once:
  char x[] = "9999";
  char y[] = "123456";
  
  Serial.begin(9600);
  while(!Serial);
  
  Serial.println(var1);
  int a = sizeof(var1);
  Serial.println(strlen(var1));
  Serial.println(a); // 7
  
  strcpy(var2, x);
  Serial.println(var2);
  int b = sizeof(var2);
  Serial.println(strlen(var2));
  Serial.println(b); // 7 (5)
  
  strcpy(var3, y);
  Serial.println(var3);
  int c = sizeof(var3);
  Serial.println(strlen(var3));
  Serial.println(c); // 7

  strcpy(var4, x);
  strcat(var4, y);
  Serial.println(var4);
  var4[sizeof(var4) - 1] = '\0';
  // or : var4[10] = '\0';
  int d = sizeof(var4);
  Serial.println(strlen(var4));
  Serial.println(d); // 11
   
}

So, what are you trying to determine: The size of the allocated memory or the length of the array?

18

PaulS:
No, it wouldn't. strcpy() and strcat() takes strings (not just char arrays) as arguments. The input strings are NULL terminated. The output string is NULL terminated.
There is no downside. The size of a string is determined by strlen(), not sizeof().

Ok so I just learnt a few new things viz.:

  • No need to null terminate a concatenated array
  • sizeof() will give me the capacity of an array + 1
  • strlen() will give me the number of elements in an array

Thanks!

19

econjack:
So, what are you trying to determine: The size of the allocated memory or the length of the array?

The code was just an experiment I was doing.
I was trying to determine "how" to add null termination to a concatenated string.
But it is a moot point as explained by PaulS.

20
  • sizeof() will give me the capacity of an array + 1
  • strlen() will give me the number of elements in an array

Actually, it should be:

sizeof() gives me the number of bytes allocated to the data item.
strlen() gives me the number of characters used in an array.

I'm not sure why the "+1" is in your first conclusion. If you define an array as:

char myName[10];

You have defined 10 elements which run from 0 through 9, so sizeof() returns 10.

Your second conclusion is a little off because you can do this:

char myName[10];
strcpy(myName, "Fred");

which causes strlen(myName) to return 4, not 10.