srtlen() problem "without Serial.println();"

Projects Programming
1

Hiya guys,

The subject description is a little weird, but i'll try to explain it better.
I have build a small function to split a char array with delimiters by using strtok(). As known, strtok() ajusts the input array. To prevent this, i used a simple loop to copy the input array to a temporary string using the strlen() function. However, this does not work. During testing I found that when using Serial.println() to show the strlen() output all works well.

An explanation by example:

This goes wrong

char** strSplit(char data[], char delimiters[]) {  
  size_t ldata = strlen(data);
  char tdata[ldata+1];
  for (int i = 0; i <= ldata; i++)
    tdata[i] = data[i];    
  byte cnt = 0;
  char* ptr = strtok(tdata, delimiters);  
  char** result = new char*[10];  
  for (int i = 0; i < 10; i++)     
    result[i] = "";
  while(ptr != NULL) {
    result[cnt] = ptr;
    cnt++;
    ptr = strtok(NULL, delimiters);
  }  
  return result;
}

And now it's OK:

char** strSplit(char data[], char delimiters[]) {  
  size_t ldata = strlen(data);
  Serial.println(ldata);             // Add this line....
  char tdata[ldata+1];
  for (int i = 0; i <= ldata; i++)
    tdata[i] = data[i];    
  byte cnt = 0;
  char* ptr = strtok(tdata, delimiters);  
  char** result = new char*[10];  
  for (int i = 0; i < 10; i++)     
    result[i] = "";
  while(ptr != NULL) {
    result[cnt] = ptr;
    cnt++;
    ptr = strtok(NULL, delimiters);
  }  
  return result;
}

I am unable to find the problem myself, I'm not a professional programmer. Ofcourse there's the possibility of some other code interfering. The whole code can be found here:

If someone can tell me what I am doing wrong it would be great.
Thanks in advance!

2

i used a simple loop to copy the input array to a temporary string using the strlen() function.

It would be easier to use strcpy()

3

The problem is you are returning pointers that point to stack addresses. These addresses become invalid once you return from the function.

4

You're returning pointers to the local array tdata, you can't do that. The reason that it sometimes works is because you're invoking undefined behavior.

You can use the pointers returned by strtok to calculate the offsets of the delimiters, but you can't return them, as they point to characters that no longer exist once you return from the function.

To be honest, I would rethink the design of the entire function. Returning a raw pointer that was manually allocated by new is very bad practice, because it's a memory leak waiting to happen. What is the point of using strtok if you're not using the result? The whole point of strtok is that it writes null terminators so that you can use the separate tokens.

Pieter

5

If you want to maintain the original string and don't want to duplicate the string, you can use strchr instead of strtok. It's a bit more work but you can maintain your original while working on it.

6

Awesome, thanks for your answers