String vs string

Hello

I just read the article The Evils of Arduino Strings | Majenko's Hardware Hacking Blog about the advantages of strings over Strings. The author explains the fragmentation of the heap by Strings and how to work with Strings and strings. But it is still not clear to me, why the fragmentation does not happen with strings? They also need to use memory. And if you delete or change the size of a string, it will also cause fragmentation. Furthermore, he explains the advantages of pointers for function arguments instead of copies. But this can be done with both, strings and Strings. Do strings maybe use another, less valuable or bigger memory segment?

To extend a little bit about the background of my question: I currently attempt to save the response of my WiFi server. Since the length of the response is not fixed, I look for a way to save it. Can you recommend an approach? My ideas are a dynamically allocated String, which is obviously not recommended, or a very long string, in order to handle even long server responses.

Thank you in advance.

Fragmentation can occur when you're using dynamic memory allocation and then releasing it. String objects use memory allocation all the time. You can allocate memory to put C strings in, but you don't have to. They may be global or exist ephemerally on the stack.

Since String operations involve behind the scenes allocations, it's easy to write code that makes Swiss cheese of your heap without realizing it. If you want to achieve the same disaster with C strings, you have to be explicit about it.

String objects have their place, but in my view, that place is far away from an Arduino with limited RAM.

As to saving wifi results, try not to. Parse what you need as you receive it if possible. If it isn't, define a properly sized static buffer and check that you don't exceed its limits.

You can use the PString-library which brings back a part of the comfort you have with Strings.
PString is safe to use. It does not "cheese-up" your heap

best regards Stefan

Why do you need to save the response at all? Usually you can just send the data consecutively and it will be recognized at the receiving end, once the delimiting symbols have been seen. That general principle applies both to serial and http connections. Often, programmers falsely believe that the entire message must be concatenated and sent in one big block, that is not the case.

On which device are you sending, and which one receiving? Are you assembling a received message, or assembling a message for transmission?

wildbill:
Fragmentation can occur when you're using dynamic memory allocation and then releasing it. String objects use memory allocation all the time. You can allocate memory to put C strings in, but you don't have to. They may be global or exist ephemerally on the stack.

Since String operations involve behind the scenes allocations, it's easy to write code that makes Swiss cheese of your heap without realizing it. If you want to achieve the same disaster with C strings, you have to be explicit about it.

String objects have their place, but in my view, that place is far away from an Arduino with limited RAM.

As to saving wifi results, try not to. Parse what you need as you receive it if possible. If it isn't, define a properly sized static buffer and check that you don't exceed its limits.

Thank you. So using strings it is just more obvious, when I fragmentate my memory. But both, strings and Strings, are capable of doing so.

aarg:
Why do you need to save the response at all? Usually you can just send the data consecutively and it will be recognized at the receiving end, once the delimiting symbols have been seen. That general principle applies both to serial and http connections. Often, programmers falsely believe that the entire message must be concatenated and sent in one big block, that is not the case.

On which device are you sending, and which one receiving? Are you assembling a received message, or assembling a message for transmission?

I use an ESP8266 with the ESP8266Wifi and ESP8266WebServer libaries. But if I think about it, you are right. I actually don't need to save the whole response.

johnson3000:
Thank you. So using strings it is just more obvious, when I fragmentate my memory. But both, strings and Strings, are capable of doing so.

No, strings use the stack. They are managed by the program instruction. Strings, use the heap and not managed.

I think garbage collection is better on rhe ESPs than on the AVRs (as in, ESPs do garbage collection), but at a slight performance cost.
Certainly most of the ESP examples rely much more on Strings than the AVR ones do.

TheMemberFormerlyKnownAsAWOL:
(as in, ESPs do garbage collection)

They do? Do you have any resources on that?

Pieter

PieterP:
They do? Do you have any resources on that?

Pieter

This is the most complete information on what goes on under the hood that I have found.

Perehama:
https://www.espressif.com/sites/default/files/documentation/esp32_technical_reference_manual_en.pdf
https://www.espressif.com/sites/default/files/documentation/esp8266-technical_reference_en.pdf
This is the most complete information on what goes on under the hood that I have found.

Those PDFs provide a great overview of the hardware, but they don't cover anything related to the C++ implementation used.
IIRC, the C++ standard does allow for garbage collection, but I don't know of any popular implementation that provides this. Given that the ESP Arduino Cores use a GCC toolchain, I don't think the C++ or STL implementation has a GC.
So if it were available, you'd have to be able to find it as a library/module in the Cores, but I couldn't find anything, the String class on ESP8266 seems to just call realloc.

PieterP:
Those PDFs provide a great overview of the hardware, but they don't cover anything related to the C++ implementation used.
IIRC, the C++ standard does allow for garbage collection, but I don't know of any popular implementation that provides this. Given that the ESP Arduino Cores use a GCC toolchain, I don't think the C++ or STL implementation has a GC.
So if it were available, you'd have to be able to find it as a library/module in the Cores, but I couldn't find anything, the String class on ESP8266 seems to just call realloc.

I haven't done a full comparison, but to me, WString.h appears to be identical, but I know that the ESP has other ways of working with the heap to avoid fragmentation.

Perehama:
but I know that the ESP has other ways of working with the heap to avoid fragmentation.

You mean manually, right?
Because it seems to me that there's nothing in there that magically defragments the heap, which is what AWOL's reply seems to imply.

PieterP:
You mean manually, right?
Because it seems to me that there's nothing in there that magically defragments the heap, which is what AWOL's reply seems to imply.

Yes, manually, which suggests with more time, someone can write a magic wand for us all, but I haven't found any magic wand, yet.

For the ESP32, the underlaying OS, freeRTOS, does the memory management routines when freeRTOS runs loop(). If loop() is left blank, the time slice that loop gets from freeRTOS will allocate more time to memory clean up.

When I use strings on a ESP WROOM I use string reserve to set a reusable String buffer.

If I use a ESP WROVER, I allocate string buffers in PSRAM. I, actually place most of any variable groups, like ints, floats, S/strings, and so forth into PSRAM.

johnson3000:
Thank you. So using strings it is just more obvious, when I fragmentate my memory. But both, strings and Strings, are capable of doing so.

Well yes, but then again no. String objects use malloc to do their job, C strings don’t. So String objects by their nature may cause fragmentation, C strings won’t.

Should you wish to, you could use malloc to get space to put a string in and if you later free it, there is potential for fragmentation. However, that’s nothing to do with strings. The same would be true if you used malloc to put an array of floats in. Or a struct etc. etc.

Most Arduino code examples you will encounter won’t malloc space for strings, but every single one you see using String objects is using malloc. It’s just not explicit in the code you see. And that’s why Majenko’s article exists - Strings are evil if you don’t have ample RAM and worse, that wickedness may not become apparent until your code has been running for a while.

That said, I’ve used malloc on PCs back in the day that had less RAM than an ESP. Just be aware that there are hidden dangers.

String objects use malloc to do their job, C strings don't.

not sure what 'their job' is...

C String are just plain null terminated char arrays. It's not a class, they don't "do" anything

String is a class, and has a number of methods. Some of those methods involve indeed messing around with memory and that's where the risk is.

J-M-L:
String is a class, [...]

In OOP paradigm, class is a Keyword; it is used to create a ClassName/Tag/Data type. Therefore, String is a "data type", and it must have been created (somewhere) using class Keyword. The said data type allows us to create object. For example:

String myString; //String refers to a data type; myString is an object on which methods are applied

This is my understanding at conceptual level.

if we go into semantic, in C++ a class is a user-defined data type. So when I write "String is a class" you can literally translate into "String is a user defined data type".

In order to define such a class, it is specific to each programing language and indeed in C++ it's done using either the keyword class or struct.

Because class (begin with small c) is a keyword, I would like to use the word/spelling Class (begin with Capital C) while saying "String is a user-defined Class" to avoid confusion/mix-up with class. Union (again Capital U) is a user-defined data type which can be created using union (begin with small u) keyword. Likewise, Structure is a user-defined data type that can be created using struct keyword. Similarly, String is a user-defined data type that can be created using class keyword.

When I write a sentence I'm not using keywords and 'class' is just a common English word, like you would say object or instance, and when you write in plain English, you don't capitalize words except at the start of the sentence.

But if that rocks your boat, feel free to write the way you want. I'm sure readers will get your intent.