I am trying to parse some serial input that is coming from an arduino in my yard (sends data via xbee) into my PC (with an arduino and receiving xbee), and I would like to load each of the parts of the incoming data into separate variables. The data is transmitted to my PC every 10 min, in batches of 3 lines. For instance, the incoming text looks like this:
Each column is separated by a space (not a tab). Ideally, I would like to have each of the "BB" strings to go to variable date1, date2, and date3. For CC strings, each would go to time1, time2, time3; for DD strings to airtemp1, airtemp2, airtemp3; and EE to lt1, lt2, lt3. TT is just to show that it is the end of the line. The reason why I would like to do this is so that I can use them later on to make a custom string that uses these variables.
You could create a state machine that reads the serial input character by character and changes it state when it gets to your "BB", "CC" and "DD". At that point you would change to input-this-kind-of-data-state load a char buffer with the next characters until you see " " (space) and treat the data accordingly.
If this makes no sense to you, reading it all in a buffer and using strtok() might be easier, but not by much. Parsing data is hard.
Thank you so much PaulS & KeithRB for your feedback. This is totally new for me, so your patient introductions & help in breaking things up like this really helps me better understand how to tackle this.
Based on your input & after thinking about it a bit more, I think I will take a multi-step approach. First, I will modify my script that sends data so that it only sends 1 line every 10 minutes. I think that should help simplify things considerably. I will look for good examples that show how to load 1 line of data into a buffer.
After that, I will have to start thinking of how I can search within this buffer for the individual "starting characters" for each column (i.e.: BB, CC, etc). This way, if for some reason (interference?) I only receive part of the original line, I can still figure out what the incoming data values correspond too...
Yes, just allocate enough space for the three lines, start saving data when you see "BB" and stop after 3 "\n" s. You can then parse the data at your leisure, but you may lose data after that, if you overflow the 64 char buffer.
I decided to scale down the project to make things a bit easier. I decided to just work with only 1-line of incoming data, and load the data straight into the buffer (using start & stop data markers would be tricky in case I had xbee interference).
Since I am learning about this, I must say that I'm very proud to have been able to take a 1st solid step by making the arduino listen the incoming serial port and storing the data in a buffer. Now, my last remaining step I have is: how do I check if the data within the buffer is correct?
Below is the working code that I have so far which loads the incoming data into the buffer. I would greatly appreciate any feedback/examples of how I can obtain the final data values. Examples would be especially helpful, since I am pretty confused at this point...
/*
//Enter into keyboard to run the prog:
AA BB01/10/2014 CC12:23:25 DD32.2 EE5432 FF54.35
BUT... keep in mind that sometimes, the data received by the xbee can be truncated
due to interference, therefore resulting in something like:
12:23:25 DD32.2 EE5432 FF54.35
// Explanation:
AA stands for "start of line" (not used for now)
BB represents date (not used for now)
CC represents time (not used for now)
DD represents air temp
EE represents light
FF represents humidity
*/
void setup() {
Serial.begin(9600);
}
void loop() {
if (Serial.available() > 0) { // check for incoming serial data:
char inChar = Serial.read(); // read incoming serial data:
Serial.write(inChar); // Type the value from what you received
// SO... all the data is in the buffer :-)
/* next step(s):
1. after receiving ALL the input, separate the buffer contents based on the " " (spaces that exist in the input).
For clarity, we can call each of these "rows" (for clarity).
2. If the first two letters of any of these rows are either DD, EE, or FF, we will know what kind of
data we are dealing with, and then check to see if it is valid data just by doing some simple comparisons
to what the "formatting" should be for each of these:
a. count to check how many digits this variable has (without the 2-letter prefix)
*/
}
}
/*
if (var_prefix == "DD") {
num_ref_digits = 4; \\ (the total number of digits (without the letters) that should exist here - includes decimal)
\\ (ex: DD32.2 --> the number of digits would be 4 (32.2)
} else if (var_prefix == "EE") {
num_ref_digits = 4;
} else if (var_prefix == "FF") {
num_ref_digits = 5;
}
*/
Does the data string being sent have an end of data delimiter like a carriage return? In the below test code I added a comma as an end of data delimiter for testing with the serial monitor. I used the below data string copied/pasted in the serial monitor to do some simple parsing. You might be able to use something to get the data you want.
AA BB01/10/2014 CC12:23:25 DD32.2 EE5432 FF54.35,
//zoomkat 3-5-12 simple delimited ',' string
//from serial port input (via serial monitor)
//and print result out serial port
String readString, substring;
int loc;
void setup() {
Serial.begin(9600);
Serial.println("serial delimit test 1.0"); // so I can keep track of what is loaded
}
void loop() {
//expect a string like AA BB01/10/2014 CC12:23:25 DD32.2 EE5432 FF54.35,
if (Serial.available()) {
char c = Serial.read(); //gets one byte from serial buffer
//if (c == '\n') { //looks for end of data packet marker
if (c == ',') {
Serial.println(readString); //prints string to serial port out
//do stuff
loc = readString.indexOf("BB");
//Serial.println(loc);
substring = readString.substring(loc+2, loc+12);
Serial.print("date is: ");
Serial.println(substring);
loc = readString.indexOf("CC");
//Serial.println(loc);
substring = readString.substring(loc+2, loc+11);
Serial.print("time is: ");
Serial.println(substring);
loc = readString.indexOf("DD");
//Serial.println(loc);
substring = readString.substring(loc+2, loc+6);
Serial.print("DD is: ");
Serial.println(substring);
loc = readString.indexOf("EE");
//Serial.println(loc);
substring = readString.substring(loc+2, loc+6);
Serial.print("date is: ");
Serial.println(substring);
loc = readString.indexOf("FF");
//Serial.println(loc);
substring = readString.substring(loc+2);
Serial.print("FF is: ");
Serial.println(substring);
readString=""; //clears variable for new input
substring="";
}
else {
readString += c; //makes the string readString
}
}
}
zoomkat... this is amazing! Thank you for your help... you have no idea how helpful this was!! XD
This definitely does the trick. I made some minor changes to your original code, and added a small routine that I found for checking the bytes that follow each prefix (I added this to the DD part, just for illustration purposes). Attached is the code I have so that it may help guide others in future projects.
Thank you again for your incredible feedback... super helpful!
What an amazing (& patient) arduino community!!!
//zoomkat 3-5-12 simple delimited ',' string
//from serial port input (via serial monitor)
//and print result out serial port
String readString, substring;
int loc;
float DD_var;
void setup() {
Serial.begin(9600);
Serial.println("serial delimit test 2.0"); // so I can keep track of what is loaded
}
void loop() {
//expect a string like AA BB01/10/2014 CC12:23:25 DD32.23 EE5432 FF54.35,
if (Serial.available()) {
char c = Serial.read(); //gets one byte from serial buffer
//if (c == '\n') { //looks for end of data packet marker
if (c == ',') {
Serial.println(readString); //prints string to serial port out
//Analyze each input
loc = readString.indexOf("BB");
//Serial.println(loc);
if (loc != -1) {
substring = readString.substring(loc+2, loc+12);
Serial.print("BB: ");
Serial.println(substring);
} else {
Serial.print("BB: ");
Serial.println("NA");
}
loc = readString.indexOf("CC");
//Serial.println(loc);
if (loc != -1) {
substring = readString.substring(loc+2, loc+11);
Serial.print("CC: ");
Serial.println(substring);
} else {
Serial.print("CC: ");
Serial.println("NA");
}
loc = readString.indexOf("DD"); // Here, we look for a match of "DD" in the buffer. Therefore, loc will
// be -1 if it doesn't find a match
//Serial.println(loc);
if (loc != -1) { // If a match is found, do the following:
substring = readString.substring(loc+2, loc+7); //extract the number/bytes that follow the DD prefix
Serial.print("DD: ");
Serial.println(substring);
//Check if substring is a valid number
if (isFloat(substring)) { //Checks that there are no letters/symbols in the extracted number/bytes
//Serial.println("number is ok");
} else {
//Serial.println("number failed");
}
} else {
//Serial.print("DD: ");
//Serial.println("NA");
}
loc = readString.indexOf("EE");
//Serial.println(loc);
if (loc != -1) {
substring = readString.substring(loc+2, loc+6);
Serial.print("EE: ");
Serial.println(substring);
} else {
Serial.print("EE: ");
Serial.println("NA");
}
loc = readString.indexOf("FF");
//Serial.println(loc);
if (loc != -1) {
substring = readString.substring(loc+2, loc+7);
Serial.print("FF: ");
Serial.println(substring);
} else {
Serial.print("FF: ");
Serial.println("NA");
}
readString=""; //clears variable for new input
substring="";
}
else {
readString += c; //makes the string readString
}
}
}
// Routine for checking if string is a valid number
boolean isFloat(String tString) {
String tBuf;
boolean decPt = false;
if(tString.charAt(0) == '+' || tString.charAt(0) == '-') tBuf = &tString[1];
else tBuf = tString;
for(int x=0;x<tBuf.length();x++)
{
if(tBuf.charAt(x) == '.') {
if(decPt) return false;
else decPt = true;
}
else if(tBuf.charAt(x) < '0' || tBuf.charAt(x) > '9') return false;
}
return true;
}
