I am trying to convert a sketch that used a lot of memory doing string manipulation using the string object to something more memory-friendly.
My assumption, based on what I've read, is that Strings and their methods are more dangerous to ram than char arrays, and cstring or homebrewed methods.
In the test sketch posted below there is a String and char[] verion of the same function with a helper freeRam() function found on this forum to report remaining ram. The String version actually performed better.
so my question is: How should I interpret this result in terms of finding strategies to make my larger sketch for optimized for memory?
My assumptions were that: 1) char[] is better than String generally, 2) PROGMEM could be useful, but using flash-allocated constants inside functions is very tricky, 3 passing arrays by reference to functions is not good for memory (I only read this somewhere);
Any help is greatly appreciated!
char statement[] = {"length of bench is ?Ft"};
char statement2[] = {"length of bench is ?Ft"};
void setup() {
Serial.begin(9600);
}
void loop() {
Serial.println(replacewithValue(statement, 44));
Serial.println(replacewithValue2(statement, 55));
delay(3000);
}
// String version:
String replacewithValue(String theString, int value) {
String stringA = theString, stringB = theString;
int start = 0, ending = 0;
freeRam();
for (int i = 0; i < theString.length(); i++) {
if ( theString.charAt(i) == '?' ) {
start = i;
break;
}
}
for (int i = theString.length(); i > 0; i--) {
if ( theString.charAt(i) == '?' ) {
ending = i + 1;
break;
}
}
freeRam();
stringA.remove(start);
stringB.remove(0, ending); freeRam();
String rtn = stringA + value + stringB; freeRam();
return rtn;
}
//char array version:
String replacewithValue2(char in[], int value) {
int a, b = 0;
char A[300]; char B[300]; char C[300];
freeRam();
for (int i = 0; i < strlen(in); i++) {
if (in[i] != '?') {
A[i] = in[i];
} else {
A[i] = '\0';
a = i;
break;
}
} freeRam();
for (int i = a + 1; i < strlen(in); i++) {
B[i - (a + 1)] = in[i];
b++;
}
B[b] = '\0';
freeRam();
sprintf(C, "%s%d%s", A, value, B);
freeRam();
return C;
}
//ram monitor function I found on this forum
void freeRam () {
extern int __heap_start, *__brkval;
int v;
int rtn = (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
Serial.println(rtn);
}
OUTPUT:
2220
2220
2220
2168
length of bench is 44Ft
1411
1411
1411
1401
length of bench is 55Ft
I changed the char[] buffers from 300, to 24, the minimum for size for the statement being passed. according to freeRam() this puts the two methods at a tie, with the String function having a big advantage of being flexible given it's input. I understand this is pretty limited test. I will repost with my full sketch after I edit it ti not contain some urls related to my job.
yes it's a function from a larger sketch that substitutes a value for a '?' char in a statement. the returned string also gets manipulated in that larger sketch and added to another string, but the statement and other strings can change. I thought it would be a good function to test different memory saving strategies. I was just surprised that the char version wasn't any better, but maybe in the local scope of a function it doesn't matter or that an isolated test function can't simulate memory problems that would accumulate in a larger sketch.
oraz:
My assumption, based on what I've read, is that Strings and their methods are more dangerous to ram than char arrays, and cstring or homebrewed methods.
...
My assumptions were that:
char[] is better than String generally
PROGMEM could be useful, but using flash-allocated constants inside functions is very tricky
passing arrays by reference to functions is not good for memory (I only read this somewhere);
RAM is not endangered by any use, but you can kill it with overvoltage.
RAM does not care what its bits represent.
RAM usage and amout of contiguous RAM available after 20 days of operation is a different story.
its like saying "men are better than women", well, it depends
PROGMEM saves RAM usage and is only a little bit more complex to handle
passing arrays by value is costly as to RAM usage, passing them by reference is like passing an int
So I think you should correct your assumptions.
No need to hurry if you are heading the wrong way.
@ robtillaart your code does perform a bit better than the string method. The difference is note huge though.
That is probably one of the design goals of dynamic memory allocation.
Yeah, I think I had some wrong impressions about ram space saving techniques based on forum threads that said it's a bad idea to use Strings when you can use char arrays. But in the scope of a local variable there doesn't seem to be much difference, and you have the benefit of not hoping your buffer length is small, yet safe. I tried using strlen(), but it was giving me problems. So I guess the string vs char[] points are more about strings created on a global or main loop scope.
My application at http://www.2wg.co.nz/ makes extensive use of Strings and very limited use of char arrays. I think it proves that you do not have to avoid Strings to build a robust fully featured Arduino application.
My application has been under development for two years and runs to well more than 10,000 lines of code. Over time I managed to work out how to use Strings and avoid the problems typically associated with them.
In general if you avoid global Strings (at least ones that you will be updating) and restricts String and String manipulation to within small local procedures you should be fine. String.reserve() is also recommended when you want to manipulate Strings.
Strings are dynamic memory objects, all be it objects that can be dynamically resized through use and can fragment your heap area. It is OK to use other dynamic objects such as ethernetclient and File objects - they too need to be used with care.
My application seems to run continuously without fragmenting free heap memory (which is the problem that Strings can create) - I have SRAM checking controls that keep a close watch on SRAM usage - the stack, the heap, free heap, free SRAM and the fixed data area.
CatweazleNZ:
you do not have to avoid Strings to build a robust fully featured Arduino application.
I think the main reason String got such a bad name was a bug in avr-libc 1.6.4's malloc which caused a memory leak. This was fixed in Arduino IDE 1.0.5 I believe.
pert:
I think the main reason String got such a bad name was a bug in avr-libc 1.6.4's malloc which caused a memory leak. This was fixed in Arduino IDE 1.0.5 I believe.
I understand that however it is still easy to use Strings badly and that gives Strings a bad name because of the difficulties people get into when they do not understand the complex issue of heap fragmentation and its causes.
It is also not helpful (to the cause of Strings and the benefits they can bring to a project) that the overall message that Strings are bad is still often promoted on this forum.
Strings are typically easier to use for inexperienced programmers and they are a key target of the Arduino market.
And if Strings were indeed a bad idea they would be deprecated and eventually removed from the Arduino environment.
I've been watching this thread with increasing discomfort:
...The String version actually performed better.
...That is probably one of the design goals of dynamic memory allocation.
...So I guess the string vs char[] points are more about strings created on a global or main loop scope.
Expecting a beginner to evaluate different approaches is ludicrous. robtillaart tried to point this out, in a gentle way, with replacewithValue3.
I think Catweazle has summed it up nicely:
it is still easy to use Strings badly
people do not understand the complex issues
Strings are typically easier to use for inexperienced programmersThese are the reasons I would never recommend String to a beginner. They would be better off in the long run by learning about arrays and why "strlen gave them problems". The beginner that starts with String and bangs out his project quickly, with great deal of satisfaction, eventually runs into The Problem.
Similarly, I would never expect to see an expert use String. They have (usually) learned that there are no net benefits to malloc/free (related thread here and countless other places on the intertubes). The "Easy" benefit is far outweighed by "Easy to use badly" and "Complex issues". In any embedded environment, predictability is paramount. Although it is possible to use Strings correctly (i.e., predictably), the expertise and care required to do so push (most) developers to other solutions. I especially encourage solutions that do not require a contiguous array of bytes for the entire string: Most of the OP's string should be in PROGMEM, not RAM.
And if Strings were indeed a bad idea they would be deprecatedOnly in a Star Trek Universe do bad ideas die. In this universe, Strings live on.
I really haven't found anybody that seemed to have an in depth knowledge of how the String functions actually perform dynamic allocation in the memory registers. If the String functions bit/byte shift the memory registers to defragment them when locations are freed up, then there is no end result fragmentation. It should be possible to shift the bytes in the registers to close the gaps, keeping the same byte count for the remaining allocated memory locations, and only update the indexes for the shifted allocations. By old school design, c-strings will probably have empty/wasted memory locations most of the time in the c-string memory locations. I guess the real answer lies at the machine level operations of the memory registries.
As far as I know when Strings are allocated they take the smallest free heap block available or expand the heap if there is not a large enough contiguous space. The same applies when you expand a String.
When any dynamic object is freed it become a block of free heap space, but if it was the last block on the heap then heap size is reduced instead.
Objects are not moved in the heap once allocated - this is what gives rise to free heap blocks within the heap when other objects or String areas are released.
When two adjoining blocks of free space occur they are merged into one larger free space block.
If you have no global dynamic objects or Strings and instantiate no dynamic objects or Strings in loop() then at the start of each iteration of loop() you will have a zero length heap and cannot suffer the slow death of a badly fragmented heap. Any dynamic objects or Strings created in procedures called by loop() should be fully released by the time program control gets back to the loop().
I do create ethernetclient objects that last for two seconds and multiple iterations of the loop(). But once I "stop" them and get back to loop() my heap is minimised (almost zero length) with no chunks of free space.
The CatweazleNZ story makes most sense to me because when I have a pointer to an object (or to inside an object) and the object is moved to fill a gap the pointer is pointing to something wrong.
when I have a pointer to an object (or to inside an object) and the object is moved to fill a gap the pointer is pointing to something wrong.
Welcome, Pandora, to the Hopeless world of Dangling References, Garbage, Garbage Collection, and the numerous techniques for dealing with them. malloc, the all-giving memory function, inevitably opens this jar. In Java, reference management is part of the language. Ironically, Java code collections can be stored in .JAR files.
