How to split a string with space and store the items in array?

how did i know you would point this out

Easy, you know I am a big fan of robust programming.

we had a course on this at qualcomm to deal with hackers sending extra long data into the phone attempting to cause it to reveal internal information.

the need was to protect against diabolical messages. we didn't need to do this for internal code.

In that case, you would cease your "String" crusade and advice ppl to use statically allocated buffers which would allow a transparent memory usage that never changes during runtime.

1 Like

How is that connected to calling a class a class?

I must admit I started out as a String skeptic, which is why I put all the work into the SafeString library which is based on statically allocated char[ ]s

But as I looked into Strings more closely to produce an example of why they were 'so bad' I found they were not that bad. Hence the Taming Arduino Strings tutorial.

If I have a crusade, it is for code that won't crash. It is a bu...er to try and debug a sketch that is crashing as this user discovered
https://forum.arduino.cc/t/use-of-string-char-causes-arduino-code-to-restart/

Strings have special place in the Arduino Framework. They are the only way to process text that is covered by the Arduino Language Reference so they are not going to go away.
This is because they are very easy to use without crashing your sketch on small memory AVR boards like the original UNO. And boards like the ESP32 use them all over the place in their WiFi libraries, so you cannot just ignore them.

That's why I am confused when people say that they are creating class(es) (lowercase c). If they would say that they are creating Classes (uppercase C)/data types, I would understand that they are creating "used-defined data types" for onward creation of objects.

That does not make any sense, at least to me.

An ESP32 has a lot more memory than a poor AVR Arduino.

@drmpf Consider a sketch running on an AVR based board which uses multiple instances of "String" for processing text, incoming commands from a serial connection or something. The "String" variables are concatenated in an overlapping manner which causes memory fragmentation. At some point, after the sketch has been running great for hours, days or even months, it is no longer possible to expand/re-alloc the internal buffer of a "String" variable in order to perform a concatenation - even though the combined memory usage of all "String" variables is only half of the memory available when the sketch was started.

Now what happens is that without any warning the "String" variable is setting itself to zero-length before the concatenation is performed and this leads to a garbled command not understood by the sketch. This again causes unwanted behaviour from which the sketch may never recover without a hard reset.

I my world this would be a malfunction (or crash) caused by the usage of the "String" variable. Had the programmer used a statically allocated buffer to begin with, this would not had happened. The "String" class requires expert knowledge to "tame", but its ease of use appeals to beginners and this is IMHO the problem with the "String" class. Oh, sorry.. I meant data type, not class.. Friggin' BEEEP BEEEP and a BEEEP in the BEEEPING BEEEP! :stuck_out_tongue:

1 Like

True, What is your point?
The AVR has a better malloc routine that makes Strings safe from crashes.

A garbled output will still allow for debugging statements to work.
Buffer overflow will just crash.
If you wan to use char[ ], my SafeString library, protects against buffer overflows, NULL pointer, etc, and has great error messages.

@Danois90 Can you post an example of String problem you describe so we can all see what is happening.

Thanks!

Even though I do not feel obliged to prove a fact, here is a sample for you. It runs for some seconds on a Nano whereafter it craps out. Please do not educate me about all the wrong-doings, and how this should have been done - this is ONLY to prove a fact.

Had there been used libraries which would have required a chunk of the free memory, this sketch would have crapped out even faster and with shorter strings.

/*
 * This sketch is created to prove a point, not to start a debate about how much 
 * if wrong with it.
 */

//If you want manual input, comment out the next line
#define SIMULATE_INPUT

//If you want to waste more of your time, comment out the next line
#define SPEED_KILL

uint16_t initialFree;
String seq, inp;
uint32_t base = 0, num = 1;

// Used to track memory usage
int freeRam ()
{
  int v;
  extern int __heap_start, *__brkval;
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}

bool validate()
{
  inp.trim();
  bool valid = inp.equals(seq);
  if (valid && (inp.length() == 0)) Serial.println("String crapped out!");
  else if (valid) Serial.println("You are correct!");
  else Serial.println("You are wrong, try again!");
  return valid;
}

String next_sequence()
{
  String res = "";
  for (uint32_t i = 0; i < num; i++)
  {
    res += String(base) + String(",");
    base++;
    #ifdef SPEED_KILL
      if (base < 1000) base += 1000;
      else if (base < 10000) base += 10000;
      else if (base < 100000UL) base += 100000UL;
      else if (base < 1000000UL) base += 1000000UL;
    #endif
  }
  num++;
  return res.substring(0, res.length() - 1);
}

void setup()
{
  Serial.begin(9600);
  initialFree = freeRam();
  Serial.print("Initial free memory: ");
  Serial.println(initialFree);
}

void loop()
{
  
  seq = next_sequence();
  Serial.print("Please type: \"");
  Serial.print(seq);
  Serial.println("\"");
  
  bool ok = false;
  do
  {
    
    #ifdef SIMULATE_INPUT
      static bool hit = true;
      inp = seq.substring(0, seq.length());
      if (!hit) inp += "-";
      hit = !hit;
    #else
      inp = Serial.readStringUntil('\n');
    #endif
    
    ok = validate();
    
  } while (!ok);
  
  uint16_t sl = seq.length() + inp.length();
  Serial.print("Length of strings: ");
  Serial.print(sl);
  Serial.print(", free memory: ");
  Serial.print(freeRam());
  Serial.print(", lost memory: ");
  Serial.println((initialFree - sl) - freeRam());
  
}

Thanks for that @Danois90
When removed your memory checks and it ran just fine.
(I will have a closer look tomorrow)
When it ran out of memory you just get a continual debug output

You are wrong, try again!
You are wrong, try again!
You are wrong, try again!
You are wrong, try again!
You are wrong, try again!

So you can start adding debug prints to see what is happening. No crash

@drmpf I am considering it a crash if a sketch stops working in an unrecoverable matter - only way out of this is a hard reset. The problem with the sketch is the usage of the "String" class (ho ho ho), and at the point of failure, the "String"'s are not large enough to not fit in memory had there been used char arrays / C-strings.

@Danois90 That is a really really good example and I will be spending more time analysing it

The using Strings in this sketch does indeed exhibit the fragmentation over time. I will look at that more closely as work permits.

However as posted the sketch overflow is due to the continual increase in the number and size of numbers you are asking it to display and read in.

This method continually increases the number of 'numbers' displayed.

String next_sequence() {
  String res = "";
  for (uint32_t i = 0; i < num; i++)   {
    res += String(base) + String(",");
    base++;
  }
  num++;
  return res.substring(0, res.length() - 1);
}

Even if you limit the max num and reset it to 1 then, so that it does not overflow whatever storage class you are using, the base++ keeps increasing with each loop and so the required storage increase with each loop.
The first loop starts at "1", next time num is reset base is num larger resulting in a longer string for the same max num
Eventually you will run out of storage no matter what you do.

You are proposing using char[ ] / cstrings as the alternative. A char[ ]/cstring version of this sketch would be a useful for comparison as it would required the algorithm to be modified to make it bounded.
BUT in any case thanks again for a very simple illustration fragmenation.

This GitHub - RobTillaart/avrheap: Arduino library to investigate the avr heap could be useful.

Thanks for the link, I am using this include to printout the heap
printFreeList.h (2.6 KB)
Rob's library looks much more complete.

@drmpf I know how the code works and why it breaks. This is not about the numbers, this is about the "abusive" use of dynamic allocations used in the background by "String". Bear in mind that the combined length of the strings never exceeds the available memory:

[FREE MEMORY ------------------------------------------]

Reserving two instances of String:

[String1][String2][FREE MEMORY ------------------------]

Now String1 has to grow by one:

[Gap1   ][String2][String1-][FREE MEMORY --------------]

Now String2 has to grow by two:

[Gap1   ][Gap2   ][String1-][String2--][FREE MEMORY ---]

Now String1 has to grow by three:

[Gap1   ][Gap2   ][Gap3    ][String2--][String1----][FM]

From here on neither instance of "String" is allowed to grow anymore due to the grude LiFo type memory manager on the AVR boards. The memory gaps (fragmentation) are unusable (or lost), so any attempt to reserve any further memory or calling methods with a large stack consumption will clash with memory in use, which again would lead to malfunction or crash.