string parsing

Projects Programming
1

I am reading in a string, which will be formatted as a set of points for a laser to follow. (x,y,enable,x,y,enable...repeated for all points) Enable just tells the laser to turn on for that line segment. The string has \n at the end.
The attached code works fine on an uno for up to 4 points. When you try 5 points, it doesn't store the data properly. I am parsing the string into an array myPoints[pointNum][x,y,or en]. Code is attached with only the code for this area.
Any help would be greatly appreciated.

sercomm3.ino (6 KB)

4

Didn't save the latest version before upload..here it is without the point class. I just saved it into an array to see if that helped.

//-----Variables --------------------------
int numPoints = 0;
int myPoints[200][3];
String inputString = ""; // a string to hold incoming data
boolean debug=true;
boolean stringAvail = false;
String com1 = "";

void setup()
{
Serial.begin(19200);
//Serial.println("setup");
//----- TLV5618 SPI Setup -----
}

void loop()
{
if(debug)
{
//Serial.println("in loop");
for (int i = 0;i<numPoints;i++)
{
char mesg[100];
sprintf(mesg, "Stored point: %d,%d,%d,%d", myPoints[0], myPoints_[1], myPoints*[2], numPoints);
 Serial.println(mesg);
 delay(10);
 }
 }_
//draw_wireframe();
_ if (stringAvail)
 {
com1 = inputString;
 parseString();
 Serial.println("clearing input");
 inputString = "";
 stringAvail = false;
 }
delay(100);
}
/
* SerialEvent occurs whenever a new data comes in the*
hardware serial RX. This routine is run between each
time loop() runs, so using delay inside loop can delay
response. Multiple bytes of data may be available.
/
void serialEvent() {
 if(debug){Serial.println("serial event");}
 while (Serial.available()) {
 // get the new byte:
 char inChar = (char)Serial.read();*_

* if (inChar == '\n')*
* { *
* if(debug)*
* {*
* Serial.print("inputString: ");*
* Serial.println(inputString);*
* }*
* //parseString(inputString);*
* //inputString = "";*
* stringAvail = true;*
* }*
* else*
* {*
* // add it to the inputString:*
* inputString += inChar;*
* } *
* } *
}
void parseString()
*{ *
* numPoints = 0;*
* while (com1.length()>1)*
* { *
* myPoints[numPoints][0] = setPos(com1);*
* myPoints[numPoints][1] = setPos(com1);*
* myPoints[numPoints][2] = setPos(com1); *
* numPoints++;*
* }*
}
int setPos(String com)
{
* int x = com.substring(0,com.indexOf(',')).toInt();*
* com1 = com.substring(com.indexOf(',')+1);*
* return x;*
}

5

Just got home and tried the same code on a Mega, same results, so I am assuming it's probably not memory. (Mega has more memory than Uno, right?) Open to suggestions for a better way to parse, or store the info...or anything really.
Thx!

6

bump

7

kberck:
bump

Bumping is discouraged here. Please read the two posts at the top of this Forum by Nick Gammon on guidelines for posting here, especially the use of code tags ("</>") when posting source code files. Also, before posting the code, use Ctrl-T in the IDE to reformat the code in a standard format, which makes it easier for us to read.

If you have already posted without using code tags, open your message and select "modify" from the pull down menu labelled, "More", at the lower left corner of the message. Highlight your code by selecting it (it turns blue), and then click on the "</>" icon at the upper left hand corner. Click on the "Save" button.

8

Please use the code button </> so your code looks like this and is easy to copy to a text editor

When the \n is detected you set stringAvail = true; and you set it back to false when you have processed the data. I wonder if your serialEvent() function should check stringAvail == false before dealing with incoming data.

...R

9

My code is listed at bottom.

I tested to make sure Serial incoming wasn't a problem. Using Serial Monitor, I sent "100,100,1,200,200,1,300,300,1,400,400,1,500,500,1,600,600,1,700,700,1\n"
The serial reads the string and stores it fine. Any new serial interrupt would result in 'serial event' being printed. inputString isn't reset until parsing is fully complete, checked that as well.

Is there a better way to get Ints out of strings, like the one listed above? String length will change.
I am researching the code below, which sounded like it should work, but I am a newb, so slow going.

char *tmp;
int i = 0;
tmp = strtok(&readString[0], ",");
while (tmp) {
   ArrayKey[i++] = atoi(tmp);
   tmp = strtok(NULL, ",");
}

My current code

//-----Variables --------------------------
int numPoints = 0;
int myPoints[200][3];
String inputString = "";         // a string to hold incoming data
boolean debug=true;
boolean stringAvail = false;
String com1 = "";

void setup()
{  
  Serial.begin(19200);
  //Serial.println("setup");
  //----- TLV5618 SPI Setup -----
}

void loop()
{      
  if(debug)
  {
    //Serial.println("in loop");
    for (int i = 0;i<numPoints;i++)
    {          
      char mesg[100];
      sprintf(mesg, "Stored point: %d,%d,%d,%d", myPoints[i][0], myPoints[i][1], myPoints[i][2], numPoints);
      Serial.println(mesg);
      delay(10);
    }
  }
  //draw_wireframe();  
  if (stringAvail)
  {    
    com1 = inputString; 
    parseString();
    Serial.println("clearing input");
    inputString = "";
    stringAvail = false;
  }    
  delay(100);
}

/*
  SerialEvent occurs whenever a new data comes in the
 hardware serial RX.  This routine is run between each
 time loop() runs, so using delay inside loop can delay
 response.  Multiple bytes of data may be available.
 */
void serialEvent() {
  if(debug){Serial.println("serial event");}
  while (Serial.available()) {
    // get the new byte:
    char inChar = (char)Serial.read();
     
    if (inChar == '\n')
    {       
      if(debug)
     {
       Serial.print("inputString: ");
       Serial.println(inputString);
     }
       //parseString(inputString);
       //inputString = "";
       stringAvail = true;
    }
    else
    {
      // add it to the inputString:
      inputString += inChar;
    }   
  }  
}

void parseString()
{     
  numPoints = 0;
  while (com1.length()>1)
  {    
    myPoints[numPoints][0] = setPos(com1);
    myPoints[numPoints][1] = setPos(com1);
    myPoints[numPoints][2] = setPos(com1);     
    numPoints++;
  }
}

int setPos(String com)
{
  int x = com.substring(0,com.indexOf(',')).toInt();
  com1 = com.substring(com.indexOf(',')+1);
  return x;
}
10 
 char mesg[100];
      sprintf(mesg, "Stored point: %d,%d,%d,%d", myPoints

    , myPoints[1], myPoints[2], numPoints);

Even before learning to post code correctly, learn to count. Take your shoes off if you need to. There is NO way that 4 integers are going to take 100 characters to print.

11

PaulS:

 char mesg[100];

sprintf(mesg, "Stored point: %d,%d,%d,%d", myPoints

, myPoints[1], myPoints[2], numPoints);



Even before learning to post code correctly, learn to count. Take your shoes off if you need to. There is NO way that 4 integers are going to take 100 characters to print.

Very interesting PaulS...so, you offer no help at all, just insults to a new member for having an array that isn't totally used? Thanks for that mate! I will take that advice to heart.

Let me add some more, so you don't have to waste your time:
WTF, you used "x" for a variable name?? How dumb are you? That is a non-descriptive name and no help at all, go back to Java!!

1...2...3...4....25. Doh! I still love you PaulS.

12

So, getting back on topic ...

You have 2048 bytes of RAM on a Uno. I advise against using 100 for just a few digits.

int myPoints[200][3];

When you try 5 points, it doesn't store the data properly.

Not surprised.

5 * 200 * 2 bytes = 2000 bytes

You have 2048 bytes of RAM.

You are using Serial.

Serial uses:

  • 34 bytes for the HardwareSerial instance (Serial)
  • 64 bytes for the Serial transmit buffer
  • 64 bytes for the Serial receive buffer
  • 4 bytes for the Serial transmit buffer head and tail pointers
  • 4 bytes for the Serial receive buffer head and tail pointers

That's 170 bytes there. So you are using 2170 out of 2048 bytes of RAM. So, it crashes.

13

"100,100,1,200,200,1,300,300,1,400,400,1,500,500,1,600,600,1,700,700,1\n"
The serial reads the string and stores it fine. Any new serial interrupt would result in 'serial event' being printed. inputString isn't reset until parsing is fully complete, checked that as well.

Is there a better way to get Ints out of strings, like the one listed above?

If your serial input definitely looks like that, then

int parsedInt = 0; 
int numbers[3];
int numbersRead = 0;

void loop() {

  read nextChar off serial;

  if(it's a digit) {
    parsedInt = parsedInt * 10 + (nextChar - '0');  
  }
  else {
    numbers[numbersRead++] = parsedInt;
    parsedInt = 0;
    if(numbersread == 3) {
      // ok! I got a group of three numbers. Do something with them.
      ... whatever ...;
      numbersRead = 0;
    }
    if(nextChar == '\n') {
      // yow! End of line! this means I do the special end-of-line thing
      ... whatever ...;
    }
  }
}
14

This is a variation on PaulM's ideas, but doing away with the post-processing while looking for the newline character. It takes 3084 bytes as written:

// Test input: 100,100,1,200,200,1,300,300,1,400,400,1,500,500,1,600,600,1,700,700,1\n

int arrayKey[25];   // Whatever you need

void setup() {
  Serial.begin(9600);
}

void loop() {
  char buffer[100];  // Again, whatever you need...
  char *ptr;
  int charsRead;
  int index;

  if (Serial.available() > 0) {
    charsRead = Serial.readBytesUntil('\n', buffer, sizeof(buffer) - 1);
    buffer[charsRead] = '\0';     // Overwrite the newline with null so it's a string

    index = 0;
    ptr = strtok(buffer, ",");
    while (ptr) {
      arrayKey[index++] = atoi(ptr);
      ptr = strtok(NULL, ",");
    }

    for (int i = 0; i < index; i++) {     // Check end result...
      Serial.print(arrayKey[i]);
      Serial.print("  ");
      if (i % 5 == 0 && i != 0)           // Break it up a bit for viewing...
        Serial.println();
    }
  }
}
15

Nick,
Memory was my first suspicion, and clearly a problem on the Uno, as you described. I had already tested the same program on a mega which resulted in the same exact problem. I tried to make a more memory efficient version, doing away with mesg[100] totally, and changing my main myPoints array to only be [20][3] instead of [200][3]. Same result. But it still behaves just like a crash from a memory problem, so I did some more debugging, and found my parsing function had an issue. Using com1.length() was giving some really unexpected results, with com1.substring(... leading to memory issues.
Anyway, I fixed those problems by counting the commas while assembling the inputString. Worked great, and stores up to 6 points perfectly. On the 7th, it freezes on trying to build the inputString, without ever getting to the parse. Just freezes. Working on that now. Might end up changing the serial, so the Java program just sends one point at a time in a loop, to avoid long string parsing altogether :slight_smile:

PaulMurrayCbr and econjack,
Thanks! Will give it a shot and report back.

16

The examples in Serial Input Basics may be useful. Simple reliable non-blocking ways to receive data.

...R

17

PaulMurrayCbr and econjack,
Yes sir, that was what I was trying to do. Still looks like memory is going to be my main enemy. If storing 200 points uses that much memory (you can imagine how many points a simple circle would use to draw) then I am going to have to shift storage back to the java side. Just hope I can refresh quick enough to avoid flicker. But I will definitely upgrade the project to my Mega.

Thank you guys for the help! (and forgiving my newb board issues!)

Robin2,
Thanks, will check it out.

18

you can imagine how many points a simple circle would use to draw

Yes, I can. One. The rest are computed.

19

kberck:
I tried to make a more memory efficient version, doing away with mesg[100] totally, and changing my main myPoints array to only be [20][3] instead of [200][3]. Same result.

Glad you sorted it eventually. However using more than the available RAM was certainly a problem. I didn't know if it was the only problem, however. I like to solve one thing at a time.

20

kberck:
Anyway, I fixed those problems by counting the commas while assembling the inputString. Worked great, and stores up to 6 points perfectly. On the 7th, it freezes on trying to build the inputString, without ever getting to the parse. Just freezes.

If you want to write effective programs on a microcontroller and avoid wasting RAM for nothing, then you better stop using the "String" object class for dynamic strings. Better now than later.

You better get used to C-strings (also called null-terminated strings or char arrays) and all the powerful string and string pointer handling build into the AVR LIBC library and its string functions:
http://www.nongnu.org/avr-libc/user-manual/group__avr__string.html

Some useful functions for you might be strtok for string parsing of comma delimited strings and the atoi function from the general AVR utilities to convert a char array pointer into an integer.

The String object class and its dynamically created Strings is not useful for systems with limited RAM like small microcontrollers.