# Problems processing a character string

**URL:** <https://forum.arduino.cc/t/problems-processing-a-character-string/694619>\
**Category:** Programming\
**Created:** [February 4, 2021, 5:05pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619 "2021-02-04T17:05:41Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![underconstrained](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/underconstrained/32/225114_2.png) [@underconstrained](https://forum.arduino.cc/u/underconstrained)\
**Post date:** [February 4, 2021, 5:05pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/1 "2021-02-04T17:05:41Z")

</div>

I’m processing data sent to my Arduino Uno that is manually entered via the com port. The data is stored in a character string and by using pointers, strtok(), strcmp(), atoi() and atof() I’m able to process the data (if entered “correctly”) and get the instructions I need. I’m now trying to handle cases where the data is entered by an unsympathetic user.

I check the first part of the string for specific sets of characters, and if I don’t get an “acceptable” value, all the data gets ignored and the user is prompted to re-enter the data. I welcome any thoughts on how to handle this any "better" than I already have.

Here is where I need the help.

In some cases, there is a second part to the string that is supposed to get converted to a 16 bit integer. I have not figured out how to handle the 2nd part of the string if its characters do not result in integer when processed by the atoi() command. Basically, I want to ignore all of the strings that fall outside of the character range of -32768 to 32767. One possible way forward might be to “ignore” strings that contain non-numerical characters, but I’m stumped on how to handle cases where the numerical characters would fall outside the 16 bit int range, or if the first strtok() completely consumes the string.

In the code below, I’m trying to correct for the condition where the user incorrectly enters a single “B”, or a “B “, followed by nothing or something other than the character equivalent of a 16bit int.

```cpp
const byte numChars = 32;  //could be as high as 64 bytes
char receivedChars[numChars];
char tempChars[numChars];        // temporary array for use when parsing
char messageFromPC[numChars] = {0};
int integerFromPC = 0;
bool newData = false;
void setup() {
    Serial.begin(9600);
    Serial.println("type 'LIST' for valid inputs format");
    delay (1000);  
}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

void loop() {
    checkSerial();
}

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

void checkSerial() {

    static byte ndx = 0;
    char endMarker = '\n';  // '\n' is for nonprintable newline, check bottom right of serial monitor and set to 'Newline' '\r' is for 'Carriage return'
    char readChar;

    while (Serial.available() > 0 && newData == false) {
        readChar = Serial.read();  // reads a character
            if (readChar != endMarker) {  
                receivedChars[ndx] = readChar;  // writes the character to an array
                ndx++;
                if (ndx >= numChars) {  // limits the string to 32 characters
                    ndx = numChars - 1;
                }
            }
            else {  // '\n' is the Enter key, and when pressed will terminate the string
                receivedChars[ndx] = '\0'; // strings must be always terminated by null character
                ndx = 0;
                newData = true;
            }
    }
    if (newData == true) {
        strcpy(tempChars, receivedChars); // copy string to preserve original because strtok() destroys the string it works on
        parseData();
        newData = false;
    }
}

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////

void parseData() {      // split the data into its parts

    char * strtokIndx; // this is used by strtok() as an index by declaring a pointer to a memory address
    strtokIndx = strtok(tempChars," "); // get the first part of string which is terminated by null character
    strcpy(messageFromPC, strtokIndx); // copy first part of string to messageFromPC

// put this strtok in an if loop so that it is only called if the correct message is received
  if (strcmp(messageFromPC, "LIST") == 0){
      Serial.println("LIST    = returns list of all commands to serial port"); 
      Serial.println("A      = returns something"); 
      Serial.println("B      = returns an integer and expects input format 'B 1000' "); 
  }
  else if (strcmp(messageFromPC, "A") == 0){// this ignores everything after the first string
      Fonzi();  
  }
  else if (strcmp(messageFromPC, "B") == 0){// Expects an int and NEEDS ERROR CORRECTION
      strtokIndx = strtok(NULL, " ");      // this continues where the previous call left off
      integerFromPC = atoi(strtokIndx);    // convert this part to an integer
      Serial.print("B = ");
      Serial.println(integerFromPC);
  }
  else{ //this should handle all cases where the first part of the string is invalid
      Serial.println("data entered is invalid, type 'LIST' for valid inputs");
      Serial.println();

  }
}
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

void Fonzi(){
    Serial.println("Aaaaaaaaaaaaaa!");
}

```

References: I have tried to absorb as much of this thread as possible, [Serial Input Basics - updated - Introductory Tutorials - Arduino Forum](https://forum.arduino.cc/index.php?topic=396450.0) and I have borrowed heavily from example 5. Per recommendations from this forum, I have purchased a copy of “C Programming Language 2nd Edition” by Kernighan & Ritchie, but I am an impatient beast and it has not yet arrived.

I also found this link helpful to find the functions that will convert the data into other lengths of ints. [IBM Documentation](https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_73/rtref/itoi.htm)

---

<div class="post-metadata">

**Author:** ![sterretje](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/sterretje/32/391294_2.png) [@sterretje](https://forum.arduino.cc/u/sterretje)\
**Post date:** [February 4, 2021, 5:45pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/2 "2021-02-04T17:45:54Z")

</div>

You can use strtol(); below for demo purpose

```cpp
void setup()
{
  Serial.begin(57600);

  strcpy(receivedChars, "12345678aa");

  char *endptr;
  long val = strtol(receivedChars, &endptr, 10);
  if (endptr == receivedChars)
  {
    Serial.println("No digits found");
  }
  Serial.println(val);
  if (*endptr != '\0')
  {
    Serial.print("Non-digit found; after number you got: ");
    Serial.println(endptr);
  }

  Serial.println("type 'LIST' for valid inputs format");
  delay (1000);
}

```

---

<div class="post-metadata">

**Author:** ![jimLee](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/jimlee/32/214865_2.png) [@jimLee](https://forum.arduino.cc/u/jimLee)\
**Post date:** [February 4, 2021, 7:40pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/3 "2021-02-04T19:40:17Z")

</div>

To me, this is a perfect case for a parser..

```cpp
#include "lilParser.h"

 
enum commands {  noCommand,  // ALWAYS start with noCommand. Or something simlar.
                  cmdA,      // The rest is up to you. help would be a good one. Have it list
                  cmdB,      // What the other commands are, how they work and what they do.
                  };          // Our list of commands.

lilParser  ourParser;        // The parser object.

void setup() {
   
  //Serial.begin(9600);
  Serial.begin(57600);
   
  // Link typed in strings to commands.
  // These can be any string without witespace.
  ourParser.addCmd(cmdA,"A");  // Type "A"  for command A.
  ourParser.addCmd(cmdB,"B");  // Type "B"  for command B.
}

// Your loop where it parses out all your typings.
void loop(void) {

  char  inChar;
  int  command;
   
  if (Serial.available()) {                      // If serial has some data..
      inChar = Serial.read();                      // Read out a charactor.
      Serial.print(inChar);                        // If using development machine, echo the charactor.
      command = ourParser.addChar(inChar);        // Try parsing what we have.
      switch (command) {                          // Check the results.
        case noCommand : break;                  // Nothing to report, move along.
        case cmdA      : handleCmdA();  break;  // Turn the LED on (HIGH is the voltage level)
        case cmdB      : handleCmdB();  break;  // Turn the LED off by making the voltage LOW
        default        : sendHelp();    break;  // No idea. Try again?
      }
  }
}

// *******Command handlers*******

void handleCmdA() {
   
  Serial.println("Aaaaaaaaaaaaaa!");
}

void handleCmdB() {

  char* charBuff;                                
  int  value;                                    
   
  if (ourParser.numParams()) {              // If they typed in somethng past the command.
      charBuff = ourParser.getParamBuff();  // We get the first parameter, assume its the number.
      value = atoi(charBuff);                // convert this part to an integer
      free(charBuff);                        // Dump the parameter buffer ASAP.
      Serial.print("B = ");                  // Do the output thing.
      Serial.println(value);
  } else {
      Serial.println("Where is my number, huh?");
  }
}

void sendHelp() {
   
  Serial.println("LIST    = returns this list of all commands."); 
  Serial.println("A      = returns something"); 
  Serial.println("B      = returns an integer and expects input format 'B 1000' "); 
}

```

If you would like to try this way, instal LC\_baseTools from your friendly IDE library manager.

-jim lee

---

<div class="post-metadata">

**Author:** ![underconstrained](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/underconstrained/32/225114_2.png) [@underconstrained](https://forum.arduino.cc/u/underconstrained)\
**Post date:** [February 4, 2021, 8:02pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/4 "2021-02-04T20:02:31Z")

</div>

jimLee,

Thanks!

I'll need to look into this further. I'm running multiple steppers in my ultimate program and I'm trying to avoid blocking functions. This being said, this looks useful and worthy of further exploration

---

<div class="post-metadata">

**Author:** ![Deva\_Rishi](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/deva_rishi/32/227254_2.png) [@Deva\_Rishi](https://forum.arduino.cc/u/Deva_Rishi)\
**Post date:** [February 4, 2021, 9:50pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/5 "2021-02-04T21:50:35Z")

</div>

The use of atoi() has a huge disadvantage of not only simply ignoring non numeric characters, but also returning 0 in case of a 'non-value' I usually just write my own version of it and use that. Mine returns -1 in case error (any non numeric characters or out of bounds, which in my case is 0 - 255, but this is one of the easiest functions to write yourself. No need for anything complicated, just test every character to see if it is within the bounds of '0' - '9' (possibly a '-' for the first character) If not, return error value, else multiply added value by 10 and add character - '0'. If you find the null terminator or another marker, return the value.

---

<div class="post-metadata">

**Author:** ![underconstrained](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/underconstrained/32/225114_2.png) [@underconstrained](https://forum.arduino.cc/u/underconstrained)\
**Post date:** [February 4, 2021, 9:51pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/6 "2021-02-04T21:51:05Z")

</div>

jimLee,

Your parser works great so far, thanks! I still need to get under the hood a little bit further. FYI is works fine on a teensy4.0, but seems to crap out on an Arduino Nano Every

My C Programming book arrived and is super helpful regarding my original direction.

Oh, and my wife grew up on Anacortes, I know it well!

---

<div class="post-metadata">

**Author:** ![underconstrained](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/underconstrained/32/225114_2.png) [@underconstrained](https://forum.arduino.cc/u/underconstrained)\
**Post date:** [February 4, 2021, 10:44pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/7 "2021-02-04T22:44:08Z")

</div>

Deva\_Rishi,

I'm going down the rabbit hole you suggested and am making some progress. Using strlen() and isdigit() functions is proving to be helpful.

I'm now also stuck on the case where the user enters a carriage return with no data before it. This crashes the program I originally posted.

---

<div class="post-metadata">

**Author:** ![anon73444976](https://avatars.discourse-cdn.com/v4/letter/a/439d5e/32.png) [@anon73444976](https://forum.arduino.cc/u/anon73444976)\
**Post date:** [February 4, 2021, 10:48pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/8 "2021-02-04T22:48:59Z")

</div>

> underconstrained:  
> I'm now also stuck on the case where the user enters a carriage return with no data before it.

Always terminate the string as it comes in.

```arduino
receivedChars[ndx++] = readChar; // writes the character to an array
         receivedChars[ndx] = '\0';

```

... but make sure there's enough room in your string...

---

<div class="post-metadata">

**Author:** ![underconstrained](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/underconstrained/32/225114_2.png) [@underconstrained](https://forum.arduino.cc/u/underconstrained)\
**Post date:** [February 4, 2021, 11:06pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/9 "2021-02-04T23:06:12Z")

</div>

TheMemberFormerlyKnownAsAWOL,

I thought the original code I posted does precisely what you mentioned while also including a check to make sure the string is not too long. I'm still not sure what is breaking the code when the carriage return is entered with no data preceding it .

---

<div class="post-metadata">

**Author:** ![anon73444976](https://avatars.discourse-cdn.com/v4/letter/a/439d5e/32.png) [@anon73444976](https://forum.arduino.cc/u/anon73444976)\
**Post date:** [February 4, 2021, 11:32pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/10 "2021-02-04T23:32:22Z")

</div>

No, I don't think I would have posted what I did, if you had done as you said you had.

You should always ensure there is space for your terminator.

---

<div class="post-metadata">

**Author:** ![underconstrained](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/underconstrained/32/225114_2.png) [@underconstrained](https://forum.arduino.cc/u/underconstrained)\
**Post date:** [February 5, 2021, 12:07am UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/11 "2021-02-05T00:07:52Z")

</div>

Got it. If I understand correctly, my checkSerial() allows for the creation of a string receivedChar[0] of zero length. Honestly, I was just following example 4 from [Serial Input Basics - updated - Introductory Tutorials - Arduino Forum](https://forum.arduino.cc/index.php?topic=396450.0)

strtok() appears to crap out when it sees a string of zero length. I'll fix checkSerial() to prevent the creation of a zero length string.

In the meantime, I just nested the offending strtok() inside in if statement that checks to see if string length is != 0.

Cheers,

---

<div class="post-metadata">

**Author:** ![christop](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/christop/32/1013044_2.png) [@christop](https://forum.arduino.cc/u/christop)\
**Post date:** [February 5, 2021, 12:10am UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/12 "2021-02-05T00:10:11Z")

</div>

> underconstrained:  
> strtok() appears to crap out when it sees a string of zero length. I'll fix checkSerial() to prevent the creation of a zero length string.

What do you mean by "crap out"? strtok() should return a null pointer in that case.

---

<div class="post-metadata">

**Author:** ![jimLee](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/jimlee/32/214865_2.png) [@jimLee](https://forum.arduino.cc/u/jimLee)\
**Post date:** [February 5, 2021, 2:06am UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/13 "2021-02-05T02:06:01Z")

</div>

@underconstrained

You're wife grew up here?! Small world! You know the old falling down bong shop with the dragon on the side? My Workshop is in the back of that old building.

As for the parser, it was originally written for a teensy 3.2. Never had an issue running it on UNOs before. Would be interested if you could let me know what you find. Its no good if it don't behave well on them. I do have an old nano in the shop..

What part of the planet did you guys end up on?

-jim lee

---

<div class="post-metadata">

**Author:** ![drmpf](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/drmpf/32/1126438_2.png) [@drmpf](https://forum.arduino.cc/u/drmpf)\
**Post date:** [February 6, 2021, 6:37am UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/14 "2021-02-06T06:37:20Z")

</div>

Check out my tutoral [Arduino Serial I/O for the Real World](https://www.forward.com.au/pfod/ArduinoProgramming/Serial_IO/index.html)  
That includes non-blocking readers and string parsing functions that won't crash your program.

---

<div class="post-metadata">

**Author:** ![Deva\_Rishi](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/deva_rishi/32/227254_2.png) [@Deva\_Rishi](https://forum.arduino.cc/u/Deva_Rishi)\
**Post date:** [February 6, 2021, 11:16am UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/15 "2021-02-06T11:16:57Z")

</div>

> christop:  
> What do you mean by "crap out"? strtok() should return a null pointer in that case.

On an ESP strtok() may have unexpected results.

---

<div class="post-metadata">

**Author:** ![drmpf](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/drmpf/32/1126438_2.png) [@drmpf](https://forum.arduino.cc/u/drmpf)\
**Post date:** [February 7, 2021, 3:56pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/16 "2021-02-07T15:56:52Z")

</div>

Here is a safer version using Strings (that does have any memory fragmentation).

```cpp
String receiveStr;
int integerFromPC = 0;

void setup() {
  Serial.begin(9600);
  for (int i = 10; i > 0; i--) {
    Serial.print(i); Serial.print(' ');
    delay(500);
  }
  Serial.println();
  Serial.println("type 'LIST' for valid inputs format");
  // receiveStr.reserve(80); // prevent heap fragmentation not really necessary here but good pratice
}

void loop() {
  if (checkSerial()) {
  //  Serial.println(receiveStr); // echo input
    parseData();
    receiveStr = ""; // finished with this input
  }
}

bool checkSerial() {
  char endMarker = '\n';  // '\n' is for nonprintable newline,
  //check bottom right of serial monitor and set to 'Newline' or Both NL and CR
  // '\r' is for 'Carriage return'
  while (Serial.available() > 0) {
    char c = Serial.read();
    if ( c == endMarker) {
      return true;
    }
    receiveStr += c;
  }
  return false;
}

void parseData() {      // split the data into its parts
  int startIdx, endIdx;
  receiveStr.trim(); // removes any leading spaces and any trailing /r
  if (receiveStr.length() == 0) {
    // empty line skip
    return;
  }
  receiveStr += ' '; // for final indexOf;
  receiveStr.toUpperCase();
  startIdx = 0;
  endIdx = receiveStr.indexOf(" ", startIdx);

  String messageFromPC = receiveStr.substring(startIdx, endIdx);

  if (messageFromPC == "LIST") {
    Serial.println("LIST    = returns list of all commands to serial port");
    Serial.println("A       = returns something");
    Serial.println("B       = returns an integer and expects input format 'B 1000' ");
  }
  else if (messageFromPC == "A") {// this ignores everything after the first string
    Fonzi();
  }
  else if (messageFromPC ==  "B") { // Expects an int and NEEDS ERROR CORRECTION
    startIdx = endIdx + 1; // for next loop step over ' '
    endIdx = receiveStr.indexOf(" ", startIdx);
    String intStr = receiveStr.substring(startIdx, endIdx);
    integerFromPC = intStr.toInt();     // convert this part to an integer
    // returns 0 if the int is missing,  use SafeString toInt() for a better method
    Serial.print("B = ");
    Serial.println(integerFromPC);
  }
  else { //this should handle all cases where the first part of the string is invalid
    Serial.println("data entered is invalid, type 'LIST' for valid inputs");
    Serial.println();
  }
}
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

void Fonzi() {
  Serial.println("Aaaaaaaaaaaaaa!");
}

```

My [SafeString library](https://www.forward.com.au/pfod/ArduinoProgramming/SafeString/index.html) has a much more robust toInt() method that return false if it fails

---

<div class="post-metadata">

**Author:** ![christop](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/christop/32/1013044_2.png) [@christop](https://forum.arduino.cc/u/christop)\
**Post date:** [February 8, 2021, 11:22pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/17 "2021-02-08T23:22:24Z")

</div>

> Deva\_Rishi:  
> On an ESP strtok() may have unexpected results.

That's still very vague. What results do you get? Does it return a pointer to the string "banana"? That would be unexpected. (And if strtok() doesn't return a null pointer when it's given an empty string, then it would be a good idea to report that bug to the ESP developers.)

---

<div class="post-metadata">

**Author:** ![cattledog](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/cattledog/32/217816_2.png) [@cattledog](https://forum.arduino.cc/u/cattledog)\
**Post date:** [February 9, 2021, 1:12am UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/18 "2021-02-09T01:12:06Z")

</div>

> And if strtok() doesn't return a null pointer when it's given an empty string, then it would be a good idea to report that bug to the ESP developers.

The problem as I understand it is that strtok() returns a null pointer, i.e. a pointer with the value of 0 and therefore points to memory address 0. That is OK in the avr architecture, but it creates an error on the esp8622. I don't know about the esp32.

See the discussion here  
[https://forum.arduino.cc/index.php?topic=682930.15](https://forum.arduino.cc/index.php?topic=682930.15)

---

<div class="post-metadata">

**Author:** ![christop](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/christop/32/1013044_2.png) [@christop](https://forum.arduino.cc/u/christop)\
**Post date:** [February 9, 2021, 4:12pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/19 "2021-02-09T16:12:42Z")

</div>

> cattledog:  
> The problem as I understand it is that strtok() returns a null pointer, i.e. a pointer with the value of 0 and therefore points to memory address 0. That is OK in the avr architecture, but it creates an error on the esp8622. I don't know about the esp32.

Ah, then it's actually a bug in the _application_ and not in the `strtok()` function itself. The application should be written to detect a null pointer because `strtok()` is expected to return a null pointer when a string is zero length (or, to be more general, has no tokens).

---

<div class="post-metadata">

**Author:** ![drmpf](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/drmpf/32/1126438_2.png) [@drmpf](https://forum.arduino.cc/u/drmpf)\
**Post date:** [February 9, 2021, 4:22pm UTC](https://forum.arduino.cc/t/problems-processing-a-character-string/694619/20 "2021-02-09T16:22:16Z")

</div>

Use my SafeString library available from the library manager, ([detailed tutorial here](https://www.forward.com.au/pfod/ArduinoProgramming/SafeString/index.html))  
Its **stoken()** method does not have those types on 'bugs' and won't make your sketch crash.

SafeString also has detailed error messages that help you find other logic issues in your string parsing

[Next page](https://forum.arduino.cc/t/problems-processing-a-character-string/694619.md?page=2)
