Is String safe to use on a ESP8266?

Projects Programming
1

Hello everyone!

As the subject says, I'm wondering about String objects usage and program reliability on a ESP8266.
My concern is about excessive heap fragmentation that would hang the execution in the long run; which this object is well-known to cause.

However, does this micro have enough RAM to make heap fragmentation not an issue? I would like to use String objects because it makes easier to code parser functions to process HTTP headers (yes, the ESP will host a very simple web server as an interface to trigger an electric door lock via Wi-Fi) and text commands (sort of AT commands via serial Bluetooth terminal, in order to check it's current status/settings and configure the ESP when the Wi-Fi access point/router isn't working). The functions for C-styled strings are not that intuitive, you know...

I could simply declare it as a global variable and use reserve() to avoid constantly reallocating the internal buffer, but what if I need to create substrings on the fly?

Best regards!

2

I think you answered your own question. String makes programming a bit easier, but you run the risk of eventually crashing your micro. Since you are building a web server, you might want to keep it up and running and available to unlock your door.

Bite the bullet, learn the C-style functions you need and be a better programmer.

3

I agree that you should learn null-terminated c-strings. However, the ESP8266 core libraries use Strings all over the place, so it doesn't really matter if you use them moderately.

Pieter

4

blh64:
Since you are building a web server, you might want to keep it up and running and available to unlock your door.

Bite the bullet, learn the C-style functions you need and be a better programmer.

Maybe you're right, but I asked this to get some opinions.
I actually already know how to deal with C-styled strings (thanks to some RAM-lacking microcontroller called "ATmega328P", at least for mild-heavy String usage); however I wasn't sure if I can do it the "easy way", or necessarily have to do it the "hard way". I mean, granted an ESP will have more than 2K of RAM, and maybe as much as the Arduino Mega; so... why not?

PieterP:
However, the ESP8266 core libraries use Strings all over the place, so it doesn't really matter if you use them moderately.

Even many web server examples make use of String objects; it might work for a good while, but not sure how it would be in a long run.

Furthermore, I'm a bit afraid since I've read some topics where people claims their ESP randomly stops functioning (I guess due to program hang or "crash") and they have to manually reset to get it back.

5

Lucario448:
however I wasn't sure if I can do it the "easy way", or necessarily have to do it the "hard way". I mean, granted an ESP will have more than 2K of RAM, and maybe as much as the Arduino Mega; so... why not?

In other words, I'm too lazy to learn the correct way to use C strings and, since I have memory resources to waste, why not waste them? You do realize that if fragmentation goes deep enough, your program will crash regardless of resource depth, right? It's just a question of when. I think most here will agree, that will happen at the most inopportune time.

6

econjack:
In other words, I'm too lazy to learn the correct way to use C strings and, since I have memory resources to waste, why not waste them?

Let's face it, that's why the String class was invented and large quantities of RAM are made available to us. When writing a program in Python on a PC I am, effectively, using them all the time.

I only use cstrings on an Arduino because I reckon if I use the String class I will get bitten pretty quickly. Presumably the risk is somewhat lower on an ESP8266 - but, unlike a PC, it does not have an operating system to protect me from myself.

I think most here will agree, that will happen at the most inopportune time.

Always - most likely when you are trying to impress a good friend.

...R

7

econjack:
since I have memory resources to waste, why not waste them?

If there is enough, because there are many variants of the ESP8266 (at least in the board and its features). I know there are differences in flash storage, but is the RAM all the same? If it is, then is just matter of taking a look in the datasheet.

econjack:
You do realize that if fragmentation goes deep enough, your program will crash regardless of resource depth, right?

True, but I thought I could still make a reliable program using String; as long as I don't declare them locally (that creates and destroys) and don't create substrings.

Call me lazy if you want, the thing is that I don't wanna overcomplicate myself and my code; although if I have no other choice to save reliability, then I would take the C way.
Is not that hard when you know how pointers work, so is not a big deal if I have to...

If the rudimentary dynamic memory (heap) manager is quite buggy, I will believe you; otherwise fragmentation shouldn't be that pronounced when executing repetitive tasks (I think)...

Robin2:
Let's face it, that's why the String class was invented and large quantities of RAM are made available to us. When writing a program in Python on a PC I am, effectively, using them all the time.

In an enviroment with plenty of RAM, its use is actually even encouraged; most likely because that runtime enviroment has a garbage collector and it's multithreaded.

Microcontrollers can't afford such feature; since they don't have large enough RAM and can't do true multitasking (DMA doesn't count). Garbage collection will punish their overall performance pretty badly; if such thing existed in an AVR, the program would completely freeze for several seconds every time it's triggered.
That's why I call their dynamic memory manager "rudimentary", in fact I don't even know how it keeps track of all free spaces while respecting the stack.

Robin2:
Presumably the risk is somewhat lower on an ESP8266 - but, unlike a PC, it does not have an operating system to protect me from myself

That's the conflict I'm having right now. ESPs have more RAM, but I usually blame (at first glance) String when people claim random freezes.

Maybe if I pre-allocate global strings with sufficient space before anything else should do the trick; because otherwise that's exactly what I will end up doing with C-strings.

Robin2:
Always - most likely when you are trying to impress a good friend.

In my case: "when I most need it".
I can't sacrifice reliability, because I don't want to realize that the ESP froze and doesn't respond to my commands in an uncomfortable situation.

8

Lucario448:
Maybe if I pre-allocate global strings with sufficient space before anything else should do the trick;

[....]

I can't sacrifice reliability, because I don't want to realize that the ESP froze and doesn't respond to my commands in an uncomfortable situation.

The problem is the word "maybe" :slight_smile:

...R

9

Most choose to use String due to the methods available for easy manipulation that seem to be missing from cstring. The operative word here is seem. Enhance your skills and develop (or Google) macros and/or functions that provide similar functionality. In fact you can get carried away and end up with a library that offers capability beyond that of the String class.

10

A high-level String class is very useful. Using less intuitive cstring functions does not make you a better programmer.
If you reserve enough memory beforehand, and you see to it that not too many intermediate Strings are created, you won't run into trouble, especially if you have such a large heap as on the ESP.

11

I do not know that much about ESP8266, but if I would venture into using the String class, I would program my sketch to monitor the amount of free, dynamic memory very carefully. If at any point the available memory drops below 5-10%, I would "alarm the user" somehow (eg. network event, an LED lighting up).

12

Danois90:
I do not know that much about ESP8266, but if I would venture into using the String class, I would program my sketch to monitor the amount of free, dynamic memory very carefully. If at any point the available memory drops below 5-10%, I would "alarm the user" somehow (eg. network event, an LED lighting up).

The amount of free memory doesn't tell you anything about the fragmentation level.

13

The challenge is that there is no run time healer for the memory and that you don't control what the String class does behind your back

for example when you use + (concatenate) operations that will allocate a temporary StringSumHelper - even if you used plenty of pre-allocated space for the function.

So you have very limited control on what's really happening... and long running system will possibly be impacted over time, which drives developer to handle this by rebooting their system from time to time to clean up...

that being said, yes - unfortunately the String class is heavily used in many libraries (eg base64 or IPAddress (this one is plain stupid, they could have made the local buffer static and return a cString they are building and rewrite differently the isValid ones)) on the ESP8266 so all bets are already off if you rely on those libraries...

14

PieterP:
The amount of free memory doesn't tell you anything about the fragmentation level.

Testing the amount of free memory (allocateable heap) should be a viable indicator of whether or not the memory (ab)usage may cause stability problems. I found this thread and from that I assume that one will not be able to allocate more memory than the largest, free gap. If the amount of free heap is reported as 20k, and you cannot allocate a buffer of 2-5k, that would indicate that fragmentation may become an issue.

15

Danois90:
If the amount of free heap is reported as 20k, and you cannot allocate a buffer of 2-5k, that would indicate that fragmentation may become an issue.

If that happens is there anything you can do to fix it, apart from re-booting?

...R

16

Robin2:
If that happens is there anything you can do to fix it, apart from re-booting?

...R

I do not think there is any feature to "defragmentate" memory. But knowing (in code) that there is a problem evolving, makes it possible to issue a reset before the device locks up.

17

Lucario448:
True, but I thought I could still make a reliable program using String; as long as I don't declare them locally (that creates and destroys) and don't create substrings.

So, your solution is still to take the easy route, even though you know that globals are often a source of program bugs and goes against the concept of encapsulation. It seems to me your coding style is geared to working around an issue that you know exists even though a tighter approach mitigates the problem altogether. It makes even less sense since you know how to use C strings and, thus, avoid the problem. Why would you do that?

18

econjack:
So, your solution is still to take the easy route, even though you know that globals are often a source of program bugs and goes against the concept of encapsulation.

The question of whether or not global variables are a-good-thing is entirely separate from the question of using the String class.

I am quite happy to use global variables in small programs whereas I would not use the String class. The only risk with global variables is that the programmer gets confused when writing the program. The String class can bite you even if the program is perfectly written.

...R

19

I've never used one but wikipedia tells me that an ESP8266 has 80K of user RAM. So you've got a lot more space to waste on Strings and maybe you'll be ok. The trouble is, it's very hard to be sure of that. When your program is following the happy path, the heap may be stable, but if you encounter an error condition or some other code path that you didn't test adequately, there may be trouble.

If you make use of Strings, even with this much RAM, you're gambling that everything will be ok. If this is a 24/7 application, that gamble may not bite you for days, so you're never certain that your testing was enough. In the end, it comes down to how critical the application is and what the consequences will be if it crashes - you just have to decide whether to accept the risk.

20

Robin2:
The question of whether or not global variables are a-good-thing is entirely separate from the question of using the String class.

I am quite happy to use global variables in small programs whereas I would not use the String class. The only risk with global variables is that the programmer gets confused when writing the program. The String class can bite you even if the program is perfectly written.

...R

I was not passing judgement on global variables. I was arguing that he sees them as the solution to a problem that can be avoided by other, less risky, alternatives. His alternative is a "work-around" for a known problem when a simple, viable, alternative that he knows how to code and understands, exists.