Issue with strsep() not advancing to next token

Using Arduino Programming Questions
#1

Hey guys,
I’m using strsep() as part of a HTTP POST decoder function. I’ve whittled away the extraneous bits of code to leave just the bare minimum, but the bug still exists.

According to the man page, strsep() should advance the pointer past the separator in the passed character array each time it runs, so in a loop, each “token” will eventually be returned. However, that doesn’t happen for me. strsep() simply spits out the first token every time through the loop. It doesn’t move the pointer in the character array.

Any ideas why?

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
}


void loop() {
  static char receive_buffer[200];

  strcpy(receive_buffer, "ch=ho&hw=ho&12=on&13=on&10=1330");
 
  for (int i = 0; i < 6; i++) {
    processHttpElements(receive_buffer);
  }
}

byte processHttpElements(char* receive_buffer) {
  static char* field_ptr;

  field_ptr = strsep(&receive_buffer, "&");

  Serial.println(field_ptr);

}

The “ch=ho&hw=ho&12=on&13=on&10=1330” string might look a bit weird and ambiguous, but it’s like that to save RAM on the Arduino.

Thanks for any help.

-Elijah

Edit: This works…

void setup() {
  Serial.begin(115200);
}


void loop() {
  static char receive_buffer[200];

  strcpy(receive_buffer, "ch=ho&hw=ho&12=on&13=on&10=1330");
  static char* field_ptr;
  static char* receive_buffer_ptr;
  receive_buffer_ptr = receive_buffer;
  
  for (int i = 0; i < 6; i++) {

    field_ptr = strsep(&receive_buffer_ptr, "&");

    Serial.println(field_ptr);
    delay(100);
  }
}

So it seems the pointer is getting reset each time the function is called. Weird.

#2

This might help you understand what the strsep() method does.

void setup() {

  Serial.begin(115200);
  static char receive_buffer[200];

  strcpy(receive_buffer, "ch=ho&hw=ho&12=on&13=on&10=1330");

  processHttpElements(receive_buffer);
}


void loop() {
}

void processHttpElements(char* receive_buffer) 
{
  static char* field_ptr;

  while ((field_ptr = strsep(&receive_buffer, "&")) != '\0')
  {
    Serial.println(field_ptr);
  }
}
#3

Thanks for your reply Econjack, that does clarify it a bit, and is in fact how I was originally using strsep().

Unfortunately I had to rearrange my program to make it more memory efficient and to speed it up a bit. Originally I called strsep() in a while loop as you've shown, and put the resulting items into an array. I'd then loop through the array later on to match & process the HTML elements in a fairly large if/else statement.

However, that array became so big that I was using most of the available RAM on it, and the multiple strncmp()s within the HTML element if/else matching routine were rather slow. So I rearranged the program to check the if statement after each strsep(), and a separate HTML element if/else matching function relating to the page that was submitted. This sped it up a lot, since it would only check the HTTP elements that would actually occur on the particular page that was submitted, rather than checking every element that might occur across the whole site.

So the code is essentially this (pseudocode, obviously):

receive_buffer = "ch=ho&hw=ho&12=on&13=on&10=1330";

while (fields_are_remaining){
single_field = split_into_fields(receive_buffer);

html_element = split_into_elements(single_field);

if (page_submitted == "page1"){
	check_page_1_elements(html_element);
} else if page_submitted == "page2"{
	check_page_2_elements(html_element);
} //etc

So really I think my question is, how do I call strsep() within a function, and not lose its position across multiple iterations?

Thanks again.

#4

I've progressed slightly in finding out what's going on, which is the compiler seems to be creating a new pointer to the character array rather than using the one passed in. In other words the pointer is being passed by value, not by reference.

When I Serial.print the pointer within the function, it correctly returns the string without the first token, but when the function returns and I print the supposed same pointer again, it contains the first token (which is valid, as strsep swaps the delimiter for '\0'), so the pointer never gets updated.

So now it seems I've got to pass a pointer to a pointer into the function, rather than a pointer to the character array. Geez.

Edit: ...And it seems the way to do that is like so:

void processHttpElements(char* &receive_buf_ptr)

And you'd call it with

char receive_buffer[200];
char* receive_buf_ptr = receive_buffer;
processHttpElements(receive_buf_ptr);

Hope that helps someone someday..!

#5

I have not looked into strsep(), but my guess is that it works in a fashion similar to strtok(). It appears that strtok() overwrites the matches delimiter with a null so that it can treat the newly-created substring as a string. It then increments an internal pointer and looks for the next delimiter. It continues this until the ending null in the original string is read. As a result, the original string is trashed in the process. You could write your own parser and use a static char* pointer to hold the last position in the string between function calls. Keep in mind, however, that the pointer is only valid relative to the original string.

#6