Go Down

Topic: Is there a limit to how many chars you can append to a String type ? (Read 6849 times) previous topic - next topic

strykeroz

This function reconstructs a string stored in the onboard EEPROM from a known starting location up to a null is found.  Or it's supposed to...
Code: [Select]
String readEEPROMstringAt(int eeIndex) {
  // eeIndex is the start location of the string which is stored on EEPROM null terminated
  String builtString = "";
  int oldLen = builtString.length();  // used for debugging
 
  for (int i = eeIndex; EEPROM.read(i) != NULL; i++) {
    builtString += char(EEPROM.read(i));
    if(builtString.length() == oldLen) Serial.println("!!! ");  // <- debugging
    oldLen = builtString.length();
    DEBUG_PRINTLN(builtString);  // just a println wrapped in a #define
  }
  return builtString;
}


This appears to break down with larger strings, for example where I have a URL "http://pvoutput.org/service/r1/addstatus.jsp"

In the above function I added the debugging line and it gets triggered with that URL string - here's the output.
Code: [Select]
h
ht
htt
http
http:
http:/
http://
http://p
http://pv
http://pvo
http://pvou
http://pvout
http://pvoutp
http://pvoutpu
http://pvoutput
http://pvoutput.
http://pvoutput.o
http://pvoutput.or
http://pvoutput.org
http://pvoutput.org/
http://pvoutput.org/s
http://pvoutput.org/se
http://pvoutput.org/ser
http://pvoutput.org/serv
http://pvoutput.org/servi
http://pvoutput.org/servic
http://pvoutput.org/service
http://pvoutput.org/service/
http://pvoutput.org/service/r
http://pvoutput.org/service/r1
http://pvoutput.org/service/r1/
http://pvoutput.org/service/r1/a
http://pvoutput.org/service/r1/ad
http://pvoutput.org/service/r1/add
http://pvoutput.org/service/r1/adds
http://pvoutput.org/service/r1/addst
http://pvoutput.org/service/r1/addsta
http://pvoutput.org/service/r1/addstat
http://pvoutput.org/service/r1/addstatu
http://pvoutput.org/service/r1/addstatus
http://pvoutput.org/service/r1/addstatus.
http://pvoutput.org/service/r1/addstatus.j
!!!
http://pvoutput.org/service/r1/addstatus.j
!!!
http://pvoutput.org/service/r1/addstatus.j
While the logic is getting to the end of the stored string correctly, it isn't appending those last couple of characters to the String object I'm using as an accumulator.  Have I hit a limit of some kind, or why isn't it appending the s & p?

Thanks in advance, Geoff
"There is no problem so bad you can't make it worse" - retired astronaut Chris Hadfield

PaulS

You've discovered just how resource intensive String objects can be. Each time you append a character to a String, the space needed for the new String is allocated, the existing data is copied to the new location, the one character is appended, and the old location is freed. There is no way of determining of the allocation of space succeeded, or not.

You know, when you store the data in EEPROM, hoe long the data is. You should store the length, first, then the data.

When you go to read it, read the length first. Then, use malloc() to allocate some space, read the data and store it in that space, and return a pointer to that space. Forget the memory wasting String class. Don't forget to free the space when you are done.
The art of getting good answers lies in asking good questions.

dc42

I strongly advise against using the String class. It uses dynamic memory allocation and causes memory fragmentation. On a microcontroller with limited RAM, this leads to RAM exhaustion very easily. Classes like String are great on a PC with plenty of RAM and virtual memory, but IMO have no place on microcontrollers like the ones in Arduinos.

Instead of appending characters to a String, you should declare a fixed-length char array and append characters to that, taking care not to go beyond the end of the array.
Formal verification of safety-critical software, software development, and electronic design and prototyping. See http://www.eschertech.com. Please do not ask for unpaid help via PM, use the forum.

strykeroz

Thanks guys, this also explains the mad results I got when I tested some even longer strings with the same routine.  Up over 100+ and the Arduino didnt read anything into the accumulating String object, or it reset or simply hanged...and this project will be playing with longer than that in time.

Arrays of char it will be.
"There is no problem so bad you can't make it worse" - retired astronaut Chris Hadfield

zoomkat

Quote
* The String class has been reimplemented as well, by Paul Stoffregen. This  new version is more memory-efficient and robust. Some functions which  previously returned new string instances (e.g. trim() and toUpperCase())  have been changed to instead modify strings in place.


The above is from the 1.0 release notes. Seems like some of the "Each time you append a character to a String, the space needed for the new String is allocated, the existing data is copied to the new location, the one character is appended, and the old location is freed." issues may have been addressed.
Google forum search: Use Google Search box in upper right side of this page.
Why I like my 2005 Rio Yellow Honda S2000  https://www.youtube.com/watch?v=pWjMvrkUqX0

strykeroz


Quote
* The String class has been reimplemented as well, by Paul Stoffregen. This  new version is more memory-efficient and robust. Some functions which  previously returned new string instances (e.g. trim() and toUpperCase())  have been changed to instead modify strings in place.


The above is from the 1.0 release notes...<snip> issues may have been addressed.
Yes I'd read that too.  However my example is from IDE 1.0 so if you can think of other reasons for this behaviour please let me know.

Cheers, Geoff
"There is no problem so bad you can't make it worse" - retired astronaut Chris Hadfield

PaulS

Quote
The above is from the 1.0 release notes. Seems like some of the "Each time you append a character to a String, the space needed for the new String is allocated, the existing data is copied to the new location, the one character is appended, and the old location is freed." issues may have been addressed.

Well, there are release notes and there is code. The code for concatenating a character has changed. It still allocates room for one more character, for every call to += or concat() with one character supplied, and copies the old string and the new character to the new location. The difference is that it now uses realloc, instead of malloc and free, so if there is room for the internal character array to grow, it will. Otherwise, the malloc/free code is still used.

If there is not room for the realloc or malloc/free to work, the character to be added is not added, as was the case that prompted this thread in the first place.
The art of getting good answers lies in asking good questions.

strykeroz

That explains the situation perfectly.  Thanks again Paul.  Cheers ! Geoff
"There is no problem so bad you can't make it worse" - retired astronaut Chris Hadfield

Go Up