Splitting a string in to an array using delimiters.

Projects Programming
1

I am trying to write a function that takes just two parameters (A string and a delimiter)
The function splits the string at the delimiter(s) and returns an array with the substrings as the values.
The string can contain any number of delimiters.

I have tried to do this in so many ways but keep failing. the first attempt i tried converting the String Object into an std::string type then used old school C++ tokens (using strtok_r) to get the values witch worked with the testing code (example below)

  String msg = "10,20,30";
  char buf[msg.length()+1];
  msg.toCharArray(buf, sizeof(buf));
  char *token;
  char *pter = buf;
  while ((token = strtok_r(pter, ",", &pter)))
  {
    Serial.println(token);
    int numb =atoi(token) + 5;
    Serial.println(numb);
    //push token to the vector
  }
  //return vector

However as soon as I wrapped it in to a function (using an std::vectorstd::string as a return value) all i got was random characters in the serial console. witch i came to believe was because outside the function's scope the pointers become invalid.

So then I tried to create tokens using std::string::substr instead of strtok to get rid of the pointers (I don't fully understand pointers yet) however this time the serial did not even want to read the token value. after that I gave up on using the std liabery and moved back to just using String Objects.

So I tried constructing a new function using string.substring() to split the string.

void splitStringToArray(String msg, char delim)
{
  int i;
  Serial.println("length of string:" + msg.length());
  int splits = 0;
  while (i <= msg.length())
  {
    if(msg.charAt(i) == delim)
    {
      splits++;
    }
    i++;
  }
 String subStrings[splits];
 int delimIndex[splits];
 delimIndex[0]=0;
 delimIndex[1] = msg.indexOf(delim);
 i = 2;
 while (i<splits)
 {
  delimIndex[i] = msg.indexOf(delim, delimIndex[i-1] + 1);
  i++;
 }
 subStrings[0] = msg.substring(0, delimIndex[1]);
 i=1;
 while (i<strlen(subStrings))
 {
    subStrings[i] = msg.substring(delimIndex[i] + 1, delimIndex[i+1]);
    i++;
 }
 //Test
 i = 0;
 while (i<=strlen(subStrings))
 {
  Serial.println(subStrings[i]);
  i++;
 }
}

I do realize that this is a void function but I was trying to get the test to work before i turn it in to a string object array function.

However now i'm getting errors that I literally cant even comprehend what they are saying at all.
cannot convert 'String*' to 'const char*' for argument '1' to 'size_t strlen(const char*)'

and well im starting to feel like it would be easier to work out how to add gravity to the standard model of quantum physics then work out how to split a string object into substrings that are stored in an array or vector.

So I was wondering if any one could help me formulate a function to achieve my goal? Thankyou =)

2

The pseudocode for this is simple -

while string contains delimeter at position p:
   array[i++] = string.substr[0, p]
   string = string.substr[p+1]
array[i++] = string

But you have to code up the position test and the string substring operations correctly, using methods on
String, not C-string functions. Its always useful to test each operation in isolation to make sure it
does what you want (watch out for edge cases).

Note that the simple code allows empty strings in the result, this might or might not be what you want.

3

the first attempt i tried converting the String Object into an std::string type

That is NOT what the code you posted does. The char array is a string, NOT an instance of the std::string class.

The code you posted does something. You expect it to do something. If what it does isn't what you expect, your expectations are wrong. Without knowing exactly what the code does, or what you expect, we can't possibly help you correct your expectations.

4

The parse example in Serial Input Basics may help.

It is not a good idea to use the String (capital S) class on an Arduino as it can cause memory corruption in the small memory on an Arduino. Just use cstrings - char arrays terminated with 0.

...R

5

PaulS:
That is NOT what the code you posted does. The char array is a string, NOT an instance of the std::string class.

The code you posted does something. You expect it to do something. If what it does isn't what you expect, your expectations are wrong. Without knowing exactly what the code does, or what you expect, we can't possibly help you correct your expectations.

i did not post the part where i converted the object string in to a std::string because I tossed it when I gave up on using std::strings and in that context it is inralavint.

All I want is to try and write a function that splits a string in to substrings by a delimiter that can be anywhere in the string and an be a X amount of them. Then returned as an array populated with substrings. Its a generic function there's no need of me to provide example code to do that.

I only said what i said to say what I have tried to show what i already know and have tried, not what i want fixed and if i was to paste everything I have done I would have to go through my git repo and copy and past out about 15 different versions of the same thing and that is just redundant.

Long storey shot I have provided enough information to formulate the question. The examples are irrelevant and only there to provide proof of some of the attempts I have already tried.

Im sorry if this comes out as arrogant but posting questions on forums are 50/50 Some times you get great helpful people posting useful answers and other times you just get people posting everything that you did wrong in your post, and that they cant help you, read the rules, blablabla. even if you posted everything to the best of your abilities. Normally when i come across people like that i just post the question on a different site and they provide me with helpful info.
People ask questions on forum sites because there are things that they don't know and when you get people who aggressively reply back pointing out everything that you do wrong it just makes them want to curl up in a ball and not ask.

its even worse when I know i've provided all the information that is needed.

6

Robin2:
The parse example in Serial Input Basics may help.

It is not a good idea to use the String (capital S) class on an Arduino as it can cause memory corruption in the small memory on an Arduino. Just use cstrings - char arrays terminated with 0.

...R

I know that already however i'm using a library in my project that sadly requires arduino String objects as its arguments for some of its major functions and the library is crucial for the hardware that I am using so i can't swap it.

7

MarkT:
The pseudocode for this is simple -

while string contains delimeter at position p:

array[i++] = string.substr[0, p]
  string = string.substr[p+1]
array[i++] = string




But you have to code up the position test and the string substring operations correctly, using methods on
String, not C-string functions. Its always useful to test each operation in isolation to make sure it
does what you want (watch out for edge cases).

Note that the simple code allows empty strings in the result, this might or might not be what you want.

Thanyou this makes sense but im still trying to work out how to define the array. How do I define an array of string objects String[] basically just makes a charter array and not an array of string objects or at least that what it seems like it is doing I don't fully know.

Also trying to keep a track of the position of the position of the delimiters may be a challenge as well.

8

How do I define an array of string objects String[] basically just makes a charter array and not an array of string objects or at least that what it seems like it is doing I don't fully know.

String[n] DOES make an array of String objects, regardless of what you think.

Of course, that means that you must know what n is to create the proper sized array. If you don't, then an array may not be the best choice for the data structure. A linked list, a vector, etc., may be better choices.

Or, you make two passes through the String. On the first pass, you count the tokens, to know n. Then, you malloc() an array of that size. On the second pass, you populate that array.

There is NOTHING Arduino-specific in what you are trying to do.

And, if you find yourself being forced to use Strings, because that's what some lazy-assed library developer used, find a better library, or improve the one you are using.

9

Ok I have workedout my own solution to the problem I will post it for if any one in the future is having a similar issue can find it. So this function will take a string that is using commas to separate the values (AKA CSV format) and return a vector that is populated with the values.

std::vector<String> splitStringToVector(String msg){
  std::vector<String> subStrings;
  int j=0;
  for(int i =0; i < msg.length(); i++){
    if(msg.charAt(i) == ','){
      subStrings.push_back(msg.substring(j,i));
      j = i+1;
    }
  }
  subStrings.push_back(msg.substring(j,msg.length())); //to grab the last value of the string
  return subStrings;
}
10

hello ben.
thank you for the solution you posted.

this seems to be very clever and clean.
so all the time you spend was worth it!

funny that you had these "helping" and "friendly" hands in this post. which tell you that you are a noob.
but when you come up with your own brilliant thing, no one shows respect.

this forum is always the last place I use, because of these arrogant "helping" experts here.
other places are much more friendly and have better vibes.

hope this will change someday

11

So true edstoica.

And sadly, the most arrogant and unfriendly are also some of the most prolific posters, constantly prowling the forum for noobs to prey upon.

12

I've been trying to figure this out in 3 different threads on this forum. Each time, the responses are the same, banter, a silenced OP, no result. People will yell at you and tell you to google it --- forgetting that googling and trying for 2 hours was what led you to no answer. Writing code, compiling, uploading to a device, looking at serial output, takes 20 - 30 seconds each time too. In Python, Java, C#, VB, hell even javascript, this is trivial as hell to do. I'm sure it's trivial in C or C++ too. Can see why Linor is pushing Circuit Python. I'm no javascript fan, and the V8 / node.js engine has been smooshed into all sorts of - wrong - applications, but this would actually be a good fit.

commaString // 1,2,3,4
lstStrings = split(',' commaString)

lstStrings[0] = 1

13

edstoica:
funny that you had these "helping" and "friendly" hands in this post. which tell you that you are a noob.

...or that your solution may be inappropriate in a memory-limited architecture.

14 
commaString // 1,2,3,4
lstStrings = split(',' commaString)

lstStrings[0] = 1

OK. I give up. What are you on about ?
Do you have a problem splitting up a C string ?