Array scope issue

[faragher]

I'm coming from a history of hobbyist level BASIC, C#, and Python, and this is my first ground-up embedded project. I'm reasonably sure I know what the problem is but not how to solve it.

An array, defined outside the setup/main loops, is fed into strtok, and is only reading the first result. I assume this is a scope issue that has made the array read only.

char A[255] = "This|is|a|string.";
const char DELIMIT[2] = "|";
char *token;


void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  Serial.println();

}

void loop() {

    char B[255] = "Why|not|work";
    token = strtok(A, DELIMIT);
    while (token) {
      Serial.println(token);
      token = strtok(NULL, "|");
  }

   delay(5005);

}

Expected behavior: Serial outputs:

This
is
a
string

once and idles.

Observed behavior: Serial outputs "This" every five seconds (indicating once per loop).

Attempts to repair: I left my local test in the code, and changing A to B in the strtok call functions as expected (although repeats every five seconds, again, as expected). A is, effectively, a pointer, but I attempted to assign a pointer anyway, with identical results.

Copying A to B locally is undesirable as it would create excessive memory usage.

Initializing B as a return from another function is undesirable as this is a response string from a TCP/IP connection and needs to be accessed by multiple functions. I'd also like to keep it defined and not floating on my stack.

Hardware: ESP-8266 12E on a NodeMCU clone. (Known good)

So, obviously the issue is my programming and the scope. Am I missing a good way of doing this, or is using a global going to prevent me from altering the array? It seems to work on another project with an array of bytes, so I'm not ruling out an issue with a strtok edge case, since apparently it calls strtok_r behind the scenes and some people have had issues with that? I've been reading for about six hours over two days, and I'm not finding anything helpful outside my own local variable test.

Thanks.

[septillion]

If I run your code I get the expected behavior. There is a catch however. You will only see this behavior once after a reset. So you're not quick enough to open the serial monitor

And that's because strtok() alters the string you scan. While splitting it, it simply replaces all delimiter characters with a null-termination char. So second time around strtok() find "the end on the string" at the place of the delimiter. Because your B string is declared and initialized every loop, the changes from strtok() are not visible anymore after loop() ends.

A way to show this behavior:

char strA[] = "This|is|a|string.";
const char DelimStr[] = "|";
char* token;

const byte StrSize = sizeof(strA);


void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  Serial.println();

  printChars(strA, StrSize);

}

void loop() {

  //char B[255] = "Why|not|work";
  token = strtok(strA, DelimStr);
  while (token) {
    Serial.println(token);
      token = strtok(NULL, DelimStr);
  }

  printChars(strA, StrSize);
  delay(1000);
}

void printChars(char str[], byte n){
  Serial.print("Chars (in hex) in string:");
  for(byte i = 0; i < n; i++){
    Serial.print(" 0x");
    Serial.print((byte)str[i], HEX);
  }
  Serial.println();
}

[gfvalvo]

If it was a scope problem, the code wouldn't compile. It works as expected on an Uno:

This
is
a
string.
This
This
This
This
This
This
.
.
.

The second and subsequent passes through loop() only print "This" because the way strtok() replaces the "end of token" with a null terminator as described here: http://www.cplusplus.com/reference/cstring/strtok/

[faragher]

Hah! Oh, that's great. At first I thought "My serial monitor's open before the reboot, it can't be that," but you gotta check everything. I threw in a delay, and you're right, it works as intended in the first loop, I just wasn't prepared for the way it handled all subsequent loops.

Turns out the program was functioning as intended, but my debugging was in error. Computers (and amateur programmers).

Thanks, it's functioning as it should.