Avoiding Strings and ONLY using cstrings not solving memory fragmentation.

Projects Programming
1

Hi, as a result of research mainly in this forum I have been trying to avoid the use of Strings in my project.
And trying hard to port String based functions to some other handling solutions.

Although I am not complete I see that it does not produce a clean result, available memory seem to decrease with time leading to an inevitable exhaustion in the not too distant future.

I assume that every time a variable is released its chunk of memory will be returned to the system for reuse. This hole in memory may not be completely filled by the next variable candidate and a smaller hole is created that may never be used and from this phenomena, and with no garbage collection, fragmentation is inevitable.

But why so much focus on Strings, this problem will happen with any chunk of memory that is returned to the system so, on this rather simplistic view, fragmentation will be inevitable anyway.

Clarification will be highly welcomed.

Thanks
Paulo

2

pcborges:
Clarification will be highly welcomed.

Not possible without seeing your program.

...R

3

Avoiding Strings and ONLY using cstrings not solving memory fragmentation

As you do NOT have any memory fragmentation nor did you ever have a problem with memory fragmentation then of course changing from String to string did not solve your problem!

Mark

4

The reason fragmentation is a problem for char arrays and not for ints or bools is that... it isn't, both things have the same probability of screwing you up.

BUT, unlike simple types, operations on char arrays often require you to move the data somewhere else in memory (say, you add a character at the end or beginning or concatenate two of them).

That's costly and that's the reason people don't like the String class, because it hides that from the newbie. All you see is string = string1 + string2, while on the background you are allocating a new array and copying two other arrays to that place, then deleting the originals. Int additions happen in place, no extra or auxiliary memory needed.

Char arrs are also a lot bigger so it's costly from a computational POV as well, it takes more time.

The theory is that by doing all the stuff by hand using chars instead of using String, you start questioning why you're doing that, and you would try to find a way to not need to do it.

So, that's the reason people recommend char arrays over String. It also doesn't help that it was bugged. That was fixed in 2013 or so, but people still haven't caught with that yet.. :confused:

In reality, thou, the String class actually works better than you'd think, as it tries to grow the internal char array first, and then if it doesn't fit, reallocate it.

In and on itself, you wouldn't fix anything by using char arrays over String. The String class does nothing to fragment your memory by it self. I have projects running for years which use String and haven't failed yet.

As for avoiding fragmentation, as a general rule of thumb, you should allocate and de-allocate memory the least amount of times as is possible.

There's no much else to say really, allocate what you need globally, and leave it alone... or that's the goal, obviously real life can and will get more complicated than that, but on an ideal world, that's what you would like to be doing 100% of the time.

If your memory going down and down all the time, then yeah, you have a leak somewhere. Since you are allocating memory your self, I would pay special attention to that of course, but without seeing your code is impossible to tell.

5

I have followed lotsof discussions on the "String dispute" and not convinced (to the extend this dispute is presented).
On the other hand, I see a lot of examples in String manipulation (in the IDE) but not a single example for the same operations using char[].

One time I had a problem. I just changed that particular project from Uno to Mega2560
My opinion. No need to spend hours on discussion for a few $ or Euros (or whatever).
Not even just "for discussion".

6

demkat1:
and not convinced

One time I had a problem.

Even by your own evidence?

With a Mega it just takes longer to get to the problem :slight_smile:

...R

7

Robin2:
With a Mega it just takes longer to get to the problem :slight_smile:

LOL -true story; Strings are great for us beginners - eventually, we hit the fragmentation wall and will be forced convert.

8

I see a lot of examples in String manipulation (in the IDE) but not a single example for the same operations using char[]

Yes, the Arduino developers made a very bad choice in implementing the String class, and providing examples to show it off.

They never used Strings long enough to discover what a terrible mistake that was.

10

Robin2:
Even by your own evidence?

With a Mega it just takes longer to get to the problem :slight_smile:

...R

let;s put it numbers:
I have a sketch to read a serial. Max input length from transmitter is 150 characters.

10 ints for various processes
1 String for input String1 , initial value 160 spaces, then (before any read)
"".
I use String2 for String1.substring , then null it ("")
I use String2again for String1. (whatever1) , then null it ("")
I use String2again for String1. (whatever2) , then null it ("")

is anybody there believing that this sketch is going i n e v i d a b l y to crash?

12

demkat1:
is anybody there believing that this sketch is going i n e v i d a b l y to crash?

Let me put it this way --- are you sure it won't ever crash?

...R

13

Delta_G:
Depends on how you wrote that hypothetical code. If you're realloc-ing lots of different sizes of String then yes eventually it will crash.

ok, lets go one step more

char inchar = ' ';
// Strings 120 char each
String String1 = "         1         2         3         4         5         6         7         8         9          0         1         2";
String String2 = "         1         2         3         4         5         6         7         8         9          0         1         2";

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  Serial1.begin(115200);
  String1="";
  String2="";
}

void loop() {
  // put your main code here, to run repeatedly:
  ReadSerial1();
}
void ReadSerial1(void) {

  // 1 message 100 char comes every 10 sec
  if (Serial1.available() > 0) {
    while (Serial1.available() > 0)
    {
      inchar = Serial1.read();
      String1 += inchar;
      delay(5);
    }

    if ((String1.substring(1, 2).toInt()) == 1) Serial.println("1");
    else if ((String1.substring(1, 2).toInt()) == 2)  Serial.println("2");
    else Serial.println("error");

    String2 = String1.substring(0);
    Serial.println(String2);

    String2 = "";
    
    String2 = String1;
    String2.toUpperCase();
    Serial.println(String2);

    String1 = "";
    String2 = "";
  }
}
15

Delta_G:
What's the point? You can read the library code for yourself and see what happens. If it ends up calling realloc over and over with growing Strings then yes it is a problem.

yea yea
over and out

17

1 String for input String1 , initial value 160 spaces

Oh, you initially allocated 160 characters with .reserve() or something? Excellent! Perhaps your program DOES have a chance of not fragmenting the memory to unusability.
I'm sure I could write a program with dynamically allocated character arrays that had all the same problems as String-using programs, but part of the point of using char arrays is that you shouldn't use dynamic allocation at all...

I think that the actual bugs in String are gone, leaving only an incompatibility between the way that it is implemented ("wasting" as little RAM as possible) and the way that it is generally used:

String S("");

void loop() {
  int c = Serial.read();
  if (c > 0) {
    if (c == '\r') {
      Serial.println("Receieved \"" + S + "\"");
      S = "";  // reset string
    } else {
      // add the character to the string
      S += (char)c;
    }
  }
}
18

westfw:
I think that the actual bugs in String are gone, leaving only an incompatibility between the way that it is implemented ("wasting" as little RAM as possible) and the way that it is generally used.

Thus, I will not discourage beginners from using Strings; it is the stepping stone. Like myself, they'll eventually try to cram their code into a form factor with less RAM and wonder why it does not work. Only at that point, will they wonder why their code crashes. After all of these years, I finally understand what PaulS was preaching.

19

westfw:
but part of the point of using char arrays is that you shouldn't use dynamic allocation at all...

+1

...R

20

demkat1, were you zoomkat in a previous life?

21

sterretje:
demkat1, were you zoomkat in a previous life?

Dont know what is zoomkat, in any meaning

JMeller:
Thus, I will not discourage beginners from using Strings; it is the stepping stone. Like myself, they'll eventually try to cram their code into a form factor with less RAM and wonder why it does not work. Only at that point, will they wonder why their code crashes. After all of these years, I finally understand what PaulS was preaching.

it seems like a ...fragmentation in relation. I am curious to learn what PaulS (respect) preached about

22

zoomkat was a member here that was always promoting the use of String.

23

Thanks all for your feedback.

Dear Robin, although my concern arises from a couple of projects I am working at this time the reason of this post is more of a general question.

It is all a matter of cost and benefit.

Using Strings saves a lot of time and frustration (for the beginner at least) and, perhaps using alternative methods, like using micro-controllers with more competence (ESP32 for instance) can do the job at more or less the same price, not to mention "band-aid" solutions like rebooting the micro-controller from time to time.

I know there are many that will consider my paragraph above as sacrilege but it tends to be the alternative I found to be adopted by many in this very forum.

Regards
Paulo

24

pcborges:
Using Strings saves a lot of time and frustration (for the beginner at least)

Why not instead teach strcat(), strcmp(), and strcpy(); and for formatting numbers, sprintf() ?