String Array versus Char Array?

I’m not sure where to post this but it seems like a programming question to me.


I've heard that using Char Arrays uses a lot less memory than using String Arrays. I'm not clear on how Char Arrays work. I do understand how String Arrays work. So I wrote the following program using String Arrays.

```
#include <SPI.h>                          // uses Ardunio pins: 11 - MOSI, 12 - MISO, 13 - CLK, 4 - CS (Chip Select user defined)
#include <SD.h>                           // uses Ardunio pins: 11 - MOSI, 12 - MISO, 13 - CLK, 4 - CS (Chip Select user defined)
int cs = 4;                               // Chip select for SD Card. 
File Servo_Controller;                    // Text file object that will point to the Servo_Position text file. 

String Servo_Position_Text = "Test.txt";  // This will allow the opening of different text files programmatically later. 
String Servo_Data_Array[50];              // String Array to hold Servo Angle Positions and Delays.
String Servo_Angle;                       // String to hold just the Servo Angle.
String Servo_Delay;                       // String to hold just the Servo Delay.
int count = 0;                            // used to count the number of entries in a Servo_Position text file.  


void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
    while (!Serial) 
    {
      ; // wait for serial port to connect
    }
  Serial.print("Initializing SD card...");
    if (!SD.begin(cs)) 
    {
      Serial.println("initialization failed!");
      return;
    }
  Serial.println("initialization done.");

  // Open the Servo_Position file for reading:
  Servo_Controller = SD.open(Servo_Position_Text);
  if (Servo_Controller) 
    {
      Serial.println(Servo_Position_Text);

  
      while (Servo_Controller.available()) 
        {
          Servo_Data_Array[count] = Servo_Controller.readStringUntil('\n');
          count = count + 1; 
          delay(25);    // Needed this delay
        }     
      // close the file:
      Servo_Controller.close();
      } // end if
  else 
    {
      // if the file didn't open, print an error:
      Serial.println("error opening " + Servo_Position_Text);
    }

// Now work with the data using the Servo_Data_Array

  for (int index = 0; index < count-1; index = index +1)
    { 
      Serial.println(Servo_Data_Array[index]); 
      Servo_Angle = Servo_Data_Array[index].substring(0,3);  // Does not include the 4th position. 
      Servo_Delay = Servo_Data_Array[index].substring(4);    // should then go clear to the end. 
      Serial.println("Servo Angle = " + Servo_Angle);
      Serial.println("Servo Delay = " + Servo_Delay);    
    } // End for loop


} // End Setup

void loop() {
  // nothing happens after setup
}
```

This program doesn't control any servo motors yet. I'm about to add that code. 

All I'm asking about here is whether it would be better to use Char Arrays instead of String Arrays in this program. I'm not sure how to do that. 

The text file that it reads is formatted as follows:

a 3 digit servo angle, a comma, and a 4 digit delay in milliseconds. 

Here's an example of a Servo Position text file

```
000,1000
045,1200
090,0500
135,0500
180,0500
090,1000
000,2000
090,1000
```

The program will then move the servo through those angle positions for those given delays. 

I just included this information for clarity of how the program works. 

The real question here is should I used Char Arrays instead of String Arrays?

Is there anything to be gained by using Char Arrays instead of String Arrays?

If so, would anyone care to explain how this program could be altered to use Char Arrays instead of String arrays?

I'm not sure how to use Char Arrays to accomplish this same goal.

Thanks. 

P.S. If there is nothing to be gained by making the change to Char Arrays, then I'm already good to go. There's nothing wrong with the program as-is. I just heard that using Char Arrays uses a lot less programming memory. I don't know if that's true or not.

The evils of Strings page explains why Strings are problematic with Arduino and some advice on how to use strings in their place.

I read the article you pointed to. It looks like I'll need to take a college course on Char Arrays and how to use them properly.

In the meantime this program is working so until I actually start having memory problems I think I'll just leave it as is.

If someone could show how to rewrite my current very simple program using Char Arrays instead of String Arrays that would be helpful. I would also need to see an improvement on the memory usage when it's compiled to be truly convinced that it's worth the conversion.

Or, if someone could point to example programs where a single program is written out twice. Once using String Arrays and then the same program using Char Arrays. That way I could at least see how to covert between String Arrays to Char Arrays.

I think that would be the best way to learn the difference. At least for me.

By the way, my program uses an SD card that has 16GB of memory. Far more memory than I'll need.

Is there any way I can get the String Class to use the SD card for string storage and manipulation? If so that would solve any and all memory problems. 16GB is far more memory than I'll need for what I'm doing.

My program doesn't need to run at high speed, so it wouldn't matter if it takes more time to manipulate strings using SD memory.

Ok, instead of using a String array I'm using two Integer Arrays. I'm just converting the data directly to Integers upon reading it from the text file.

Saves some dynamic memory. Not much of a savings though.

Using a single String Array:
Sketch uses 13638 bytes (42%) of program storage space. Maximum is 32256 bytes.
Global variables use 1373 bytes (67%) of dynamic memory, leaving 675 bytes for local variables. Maximum is 2048 bytes.

Using two Integer Arrays to replace the single String Array:
Sketch uses 13388 bytes (41%) of program storage space. Maximum is 32256 bytes.
Global variables use 1237 bytes (60%) of dynamic memory, leaving 811 bytes for local variables. Maximum is 2048 bytes.

Robo_Pi:
Not much of a savings though.

100 bytes savings on 2048 bytes is about 5%. That's a lot.

===

Your example data shows 8 characters for each record. You can create a char array that is large enough to hold 8 characters plus the line termination (probably \r\n) plus a terminating NUL character

#define RECORDLENGTH 11

char Servo_Data[11];

...
...

You can then use readBytesUntil instead of readStringUntil.

You can clear the Servo_Data using on of the two below

memset(Servo_Data, '\0', sizeof(Servo_Data));  // clear the complete array
Servo_Data[0] = '\0';  // clear the first element

Reading the file in one go is expensive on memory; 50 records will take roughly 500 bytes of RAM (plus the overhead used by the String class). The memory statistics after compilation will not show this.

You can use strtok() to split a record on a comma and use atoi() or strtol()/strtoul() to convert text to (unsigned) chars / integers / long integers. Have a look at the updated Serial Input Basics thread to see how to use strtok() and atoi(); although written with Serial communication in mind, all principles also apply to reading from SD card (the example with the end marker can be a framework for your scenario).

@sterretje,

Thanks for the suggestions. I'll look into these tomorrow.

My program is actually working right now. But I'd like to keep the use of dynamic ram as low as possible because I hope to be adding more code as I go along. I've already figured out ways to save memory from how I was initially doing things. I'll give your suggestions a try tomorrow and see if that improves things even more.

Right now I've just discovered the seek() option for reading only one specific line at a time from the SD card. If I can do this it will help a lot. The way I'm doing it now is to take in all the information at once into an array. But if I can read just one line at a time from the SD card, then I won't need to bring in all the information at once. That would help a lot.

The more information that stays on the SD card the better. With 16GB of space on the SD card, memory is not a problem at all. I'm not about to use up 16GB of memory.

Right now I’ve just discovered the seek() option for reading only one specific line at a time from the SD card. If I can do this it will help a lot.

Are your records fixed length? If not, seek() won’t help at all.

PaulS:
Are your records fixed length? If not, seek() won't help at all.

Yes they are. See the position text file in the OP.

Each Record is current a 3 digit servo angle that ranges from 000 to 180.
Then a comma
And then a 4 digit delay in milliseconds, always using leading zeroes to keep it 4 char long.

So each record should be 8 characters long, not counting the carriage return character.

I create these text files so I have total control over the formatting of the data.

I was trying to use the seek function last night with no luck. But I didn't spend a lot of time on it. I'm sure I'll figure it out eventually.

If you don't close and open the file all the time, you can read the next record without seek.

I noticed that I missed something in an earlier code snippet. Corrected below.

#define RECORDLENGTH 11

char Servo_Data[RECORDLENGTH];

...
...

sterretje:
If you don't close and open the file all the time, you can read the next record without seek.

I wasn't even aware of this. I will be reading all entries sequentially so maybe I can just do it this way. I was closing the file after every read because I go off and do other things between reading each record. I never thought about just leaving the file open while I go off and do other things. I'll give that a shot.

Although it is feasible that I might want to open a second file during the process, in which case I would be forced to close the first file and come back to it later. If I end up in that scenario I'll need to use the seek() function. I'll study how to do it both ways. Thanks.

If I end up in that scenario I'll need to use the seek() function.

There is a pos() function that tells you where where you are, in the file. If you store the value returned by pos(), close the file, open the file, and seek() to the stored value, you will be right back where you left off reading.

PaulS:
There is a pos() function that tells you where where you are, in the file. If you store the value returned by pos(), close the file, open the file, and seek() to the stored value, you will be right back where you left off reading.

Ok that helps a lot. It's actually position() though.

When I use position() to read the position I get a record length of 2 greater than I was expecting. Apparently the carriage return is two characters long in my text files.

Once I was able to get the correct record length I was able to use the seek() function correctly. Now it's starting to make sense, except I'm still not sure why a carriage return is 2 characters long. I guess it includes an escape character?

My records on only 8 characters long. But I need to use multiples of seek(10) to get to the start of the next record.

I'm making major progress now.

Now it's starting to make sense, except I'm still not sure why a carriage return is 2 characters long. I guess it includes an escape character?

Your termination is probably a line feed and carriage return /n/r ascii 10 and 13

You can confirm that by printing out the byte values of what you read.

I figured it out, and should have known better.

My data is 8 char long.
There is 1 char for the new line '\n'
that makes 9 chars per record.

Then you need to add 1 to that to be pointing to the starting position of the next line.
That's why it needs to be 10.

So when adding to a seek() position you really need to use the record length (including the newline char) + 1 to position you at the beginning of the next line.

If seek(0) positions you at the start of the 1st record, and seek(10) positions you at the start of the second record, the records are 10 characters long - 8 for the data, one for the carriage return, and one for the line feed.

There are a few possible line terminations that you might encounter and it basically depends on the operating system. Most common are the first two below.

carriage return (\r, 0x0D) plus line feed (\n, 0x0A); used in Windows systems
line feed; used on Unix/Linux systems
carriage return; as far as I know used on old Apple Macintosh systems

There are a few more; I found Wikipedia Newline that shows them.

I have no idea what newer Macs use; you can find that out if needed by printing a line in HEX from your Arduino to a terminal program (e.g. serial monitor) as indicated by CattleDog in reply #13.

I just looked at the text file using Programmer's Notepad and every line does indeed end with a CR LF.

I don't really care how it's formatted, I'm just trying to understand how the seek() function is working.

I also like the idea of using the position() function to get the correct position. That way I won't need to know how long the records are, and if I decide to change the record size later it won't matter.

In fact, right now I'm using Windows Notepad to create these text files just for testing purposes. But later I will be generating the text files using a C# program. When I do that I will have full control over what characters I place at the end of each line. I'll probably stick with the CR LF format just to remain compatible with Windows Notepad. That way I'll be able to edit them in Notepad without messing up the format.

Robo_Pi:
I also like the idea of using the position() function to get the correct position.
That way I won't need to know how long the records are, and if I decide to change the record size later it won't matter.

That dream will not come true, position() does not help you to solve these problems.

Whandall:
That dream will not come true, position() does not help you to solve these problems.

But it already has solved this problem. That's how I found out the record is actually 10 characters long. I was thinking it was either 8 or 9.

Using position() to tell me where I'm at after having read a record tells me precisely where the next record begins.

Use it AFTER you read the record.

Look at this little line of code:

Servo_Data = Servo_Controller.readStringUntil('\n');
record_Number = Servo_Controller.position();

record_Number then contains the position of the next record to be read.

Then you can close this file, and reopen it later and use seek(record_Number), and you'll pick up right where you had left off the last time.

You won't need to know how long each line is, nor will they need to be the same length.

In fact, I just tried it and it works exactly as I expected it would.

Here the new text file:

This first record is long.
Short record.
A very much longer record to be sure
tiny rec
Lonnnnnnnng record
stop record

I open the file and read the first record "This record is long.".
then I save the record_Number using position();
then I close the file.

Then I reopen the file and read the second record using seek(record_Number);
it prints out the second line "Short record."

Then I just continue this same process and it successfully seeks the start of every record.

It works perfectly.

No need to even know how long a record is, nor is there any need for all the records to be the same length.