Get values from multiple lines of serial output

Hi there,
i am currently working on a project, that requires reading the rssi values from specific AP's so that i can later issue an HTTP GET request. For this reason i use an ESP8266 module connected through serial with Arduino Uno. Connection works well, module is responding to AT commands, but when i send the "AT+CWLAP" command problems start.

I use the serialEvent function to capture each line the "AT+CWLAP" command returns.

Serial.println("AT+CWLAP");

In order to get the whole line i use the serialEvent function :

void serialEvent() {

  inputString = "";
  while (Serial.available()) {
    // get the new byte:
    char inChar = (char)Serial.read();
    // add it to the inputString:
    inputString += inChar;
    // if the incoming character is a newline, set a flag
    // so the main loop can do something about it:
    if (inChar == '\n') {
      stringComplete = true;
    }
  }
  
}

As far i understand the serialEvent function, it reads each byte and adds it to a string. When a newline character is read, the loop() function proceeds the string. It seems preety simple, but unfortunattely is not working for me.

In fact serialEvent returns rubbish, previous outpout etc like that, unitl it hangs :

AT+CWLAP

AT+CWDHCP=1,0


OK
AT+CWJAP="*******",AT+CWLAP

busy p...

AT+CWDHCP=1,0


OK
AT+CWJAP="******",AT+CWLAP

busy p...

AT+CWDHCP=1,0


OK
AT+CWJAP="******",AT+CWLAP

busy p...


ERROR
AT+CWDHCP=1,0




busy 
p...
OK

Any ideas why is this happening?Am i missing something?
Thanks

Below is the test code :

#define ARRAY_SIZE 4

String inputString = "";              // a string to hold incoming data
String curAT;                            // current AT command
boolean stringComplete = false;  // whether the string is complete

int ap[ARRAY_SIZE];

int i=0;
int rssi;

void setup() {

  inputString.reserve(200);
  Serial.begin(115200);
 
  //testing
  for(i=0;i<ARRAY_SIZE+1;i++){
    ap[i] = 0;
  }

  Serial.println("AT+CWLAP");curAT="AT+CWLAP";//Serial.println(curAT);
  delay(4000);

}

void loop() {

  if ((stringComplete)) {
    if (inputString.startsWith("+CWLAP:")) {
      int firstComma = inputString.indexOf(',')+1;
    if (inputString.substring(firstComma+1,firstComma+6) == "ESPAP") {
        int secondComma = inputString.indexOf(',',firstComma+1)+1;
        //load rssi to array
        String rssi = inputString.substring(secondComma+1,secondComma+3);
        ap[i]=rssi.toInt();
        inputString ="";
        rssi = "";
        i+=1;
      } 
      // clear the string:
      inputString = "";
      stringComplete = false;
    }
    else if ((inputString.startsWith("OK"))&& (curAT=="AT+CWLAP")){
      //Serial.println(htmlGet());
      inputString = "";
      i=0;
    } //else if
  }  //if (stringComplete)
}

void serialEvent() {

  inputString = "";
  while (Serial.available()) {
    // get the new byte:
    char inChar = (char)Serial.read();
    // add it to the inputString:
    inputString += inChar;
    // if the incoming character is a newline, set a flag
    // so the main loop can do something about it:
    if (inChar == '\n') {
      stringComplete = true;
    }
  }
  
}

I use the serialEvent function to capture each line the "AT+CWLAP" command returns.

The serialEvent function does NOTHING magic. You could do the same thing in loop(), when there is serial data to deal with.

In fact serialEvent returns rubbish

Look again. The serialEvent() function's return type is void. It does NOT return anything.

It is pointless to try to parse the data collected by serialEvent without having a clue whether or not what it collected is reasonable. Determining whether it is reasonable, or not, is not a matter if divining anything. It IS a matter or print()ing stuff.

PaulS thanks for your reply

I used the wrong words, when describing my problem(?). What i expect after the "AT+CWLAP" is something like that:

AT+CWLAP

+CWLAP:(4,"***********",-88,"++++++++++",1,-27)
+CWLAP:(4,"******",-90,"++++++++++",1,-4)
+CWLAP:(4,"**********",-74,"++++++++++",1,-32)
+CWLAP:(2,"****",-93,"++++++++++",1,-16)

I should get each from the above lines from the serial back to the loop, which is not happening. Instead i am get the output i posted before.

The returned string is checked back to the loop

 if ((stringComplete)) {
    if (inputString.startsWith("+CWLAP:")) {

I am NOT blaming serialEvent, i am just confused.

What i expect after the "AT+CWLAP" is something like that:

THAT appears to include carriage returns, which your code recognizes as "we be done".

So, now we know what you expect. We still don't know what you actually get.

THAT appears to include carriage returns, which your code recognizes as "we be done".

Carriage returns, means line is complete, right? Code should proceed with the next line unit the last ("OK"). At least, this is what i expect...

Instead, what i actually get is lines like these:

AT+CWLAP

AT+CWDHCP=1,0


OK
AT+CWJAP="*******",AT+CWLAP

busy p...

AT+CWDHCP=1,0


OK
AT+CWJAP="******",AT+CWLAP

busy p...

AT+CWDHCP=1,0


OK
AT+CWJAP="******",AT+CWLAP

busy p...


ERROR
AT+CWDHCP=1,0




busy 
p...
OK

Instead, what i actually get is lines like these:

How can you tell what any of that mess means?

First thing to do is get the ESP8266 off of the hardware serial port, so you can use the hardware serial port for debugging.

Second thing to do is print what you get from the ESP8266 after sending it each AT command WITH AN IDENTIFIER and start and end markers:

   Serial.print("inputString: [");
   Serial.print(inputString);
   Serial.println("]");

Until then, you are just kidding your self that you think you know what you are dealing with.

Thanks PaulS, it really helps,
just rewrote the sketch without SerialEvent and using SoftwareSerial for the ESP8266 and hardware serial for debugging.

SoftwareSerial Serial1(6,7);

It's cleaner but the responce from serial is still confusing. Here is what i get after the "AT+CWLAP"

inputString: [
]
inputString: [AR)ƒWS�TZ�j

]
inputString: [OK
]

You need to post your latest program so we can keep up with you.

You may find some useful stuff in Serial Input Basics - simple reliable ways to receive data.

If the response contains a few lines of text separated by carriage-returns or line-feeds and if there is no character to mark the end of the complete message then just receive them as several separate lines. Maybe the only option is to count the number of parts until you know they have all arrived.

...R

Hi here is the revised code without serialEvent and using SosftwareSerial.

#include <SoftwareSerial.h>
#define ARRAY_SIZE 4

SoftwareSerial Serial1(6,7); //Rx,Tx

String inputString = "";         // a string to hold incoming data
String curAT;

int ap[ARRAY_SIZE];
int i=0;
int rssi;

void setup() {

  inputString.reserve(200);
  
  Serial.begin(115200);
  Serial1.begin(115200);  

  //testing
  for(i=0;i<ARRAY_SIZE+1;i++){
    ap[i] = 0;
  }

  Serial1.println("AT+CWLAP");curAT="AT+CWLAP";Serial.println(curAT);
  delay(2000);
}

void loop() {

  while (Serial1.available()) {
    delay(3);
    if (Serial.available() >0) {
    char inChar = (char)Serial1.read();
    // add it to the inputString
    inputString += inChar;
    // if the incoming character is a newline
    if (inChar == '\n') {
      Serial.print("inputString: [");Serial.print(inputString);Serial.println("]"); 
      if (inputString.startsWith("+CWLAP:")) {
        Serial.print("+CWLAP - inputString: [");Serial.print(inputString);Serial.println("]");
        //Retrieve SSID
        int firstComma = inputString.indexOf(',')+1;

        if (inputString.substring(firstComma+1,firstComma+6) == "ESPAP") {
           //Retrieve RSSI
            int secondComma = inputString.indexOf(',',firstComma+1)+1;
           //load inputString to array
           String rssi = inputString.substring(secondComma+1,secondComma+3);
           ap[i]=rssi.toInt();
           inputString ="";
           rssi = "";
           i+=1;
        } 
     }
      else if ((inputString.startsWith("OK"))&& (curAT=="AT+CWLAP")){
        i=0;
      } //else if
      inputString = "";
    } //if inChar 
    
  } //if Serial
  
  } // while

}

It should query the AP's (AT+CWLAP) and retrieve the RSSI from specific AP's (SSID="ESPAP").

Any ideas are welcome!

Robin2:
You may find some useful stuff in Serial Input Basics - simple reliable ways to receive data.

Thanks Robin2, really usefull link!

I'm not sure if Reply #8 takes account of Serial Input Basics or if you are planning to do more work on your program which will make the code in Reply #8 redundant.

And thank you for the kind words.

...R

Robin2:
I'm not sure if Reply #8 takes account of Serial Input Basics or if you are planning to do more work on your program which will make the code in Reply #8 redundant.

And thank you for the kind words.

...R

No that was an earlier sketch. I'm working on a new one, based on your examples. I'll post it asap

here is the new sketch i wrote based on example 5 from Serial Input Basics. Worked almost out of the box for me! Thanks a lot Robin2!

#define SSID "MYSSID"      // "SSID - WiFiname or name for AP" 
#define PASS "MYPASS"   // "password or pass for AP"

const byte numChars = 32;
char receivedChars[numChars];
char tempChars[numChars];        // temporary array for use when parsing

      // variables to hold the parsed data
char messageFromSerial[numChars] = {0};
char SSIDfromSerial[numChars] = {0};
int rssiFromSerial = 0;

boolean newData = false;

//============

void setup() {
    Serial.begin(115200);
      
    Serial.println("AT");//curAT="AT";//Serial.println(curAT);
    delay(1000);
    if(Serial.find("OK")){
      connectWiFi();
    }

    //Serial.println("This demo expects 3 pieces of data - text, an integer and a floating point value");
    //Serial.println("Enter data in this style <HelloWorld, 12, 24.7>  ");
    Serial.println("This demo SSID and RSSI of local APs");
    Serial.println();
}

//============

void loop() {
    //String cmd = "AT+CWLAP=\"MYSSID\"";
    String cmd = "AT+CWLAP";
    Serial.println(cmd);
    delay(10000);

    while (Serial.available() > 0) {
      
      recvWithStartEndMarkers();
      
      if (newData == true) {
          strcpy(tempChars, receivedChars);
              // this temporary copy is necessary to protect the original data
              //   because strtok() used in parseData() replaces the commas with \0
          parseData();
          showParsedData();
          newData = false;
      }
    }
}

//============

boolean connectWiFi(){
  Serial.println("AT+CWMODE=1");//curAT="AT+CWMODE=1";Serial.println(curAT);
  delay(1000);
  Serial.println("AT+CWDHCP=1,0");//curAT="AT+CWDHCP=1,0";Serial.println(curAT);
  delay(1000);
  String cmd="AT+CWJAP=\"";
  cmd+=SSID;
  cmd+="\",\"";
  cmd+=PASS;
  cmd+="\"";
  Serial.println(cmd);//curAT="AT+CWJAP";Serial.println(curAT);
  delay(2000);
  if(Serial.find("OK")){
    return true;
  }else{
    return false;
  }
}

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '+'; //expecting something like that +CWLAP:(4,"ZTE_4",-65,"2c:95:7f:49:99:34",11,-32)
    char endMarker = '\n';
    char rc;

    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

//============

void parseData() {      // split the data into its parts

    char * strtokIndx; // this is used by strtok() as an index

    strtokIndx = strtok(tempChars,",");      // get the first part - the string
    strcpy(messageFromSerial, strtokIndx); // copy it to messageFromSerial

    strtokIndx = strtok(NULL,",");      // get the first part - the string
    strcpy(SSIDfromSerial, strtokIndx); // copy it to SSIDfromSerial  

    strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
    rssiFromSerial = atoi(strtokIndx);     // convert this part to an integer

}

//============

void showParsedData() {
    Serial.print("SSID :");
    Serial.print(SSIDfromSerial);
    Serial.print(" - RSSI :");
    Serial.println(rssiFromSerial);
}

It works fine if i send a line like this:

AT+CWLAP:(4,"ZTE_4",-65,"2c:95:7f:49:99:34",11,-32)

or issue the command

"AT+CWLAP=\"MYSSID\"";

which also returns only one line.

I then changed the loop() as follows,

while (Serial.available() > 0) {
      recvWithStartEndMarkers();

in order to execute the command

"AT+CWLAP"

BUT it is still showing only one result. (There are several APs around)
Here is the outpout:

AT+CWLAP
SSID :0

 - RSSI :0
AT+CWLAP
SSID : - RSSI :0
AT+CWLAP
SSID :"MYSSID" - RSSI :-55
AT+CWLAP
SSID : - RSSI :0
AT+CWLAP
SSID :"JUNK" : - RSSI :0
 - RSSI :0
SSID : - RSSI :0
AT+CWLAP
SSID :"MYSSID" - RSSI :-58

Any ideas?Thanks

while (Serial.available() > 0) {
      recvWithStartEndMarkers();

That is all wrong. It is not how the function is intended to be used.

Give me an example of the reply that you are having trouble receiving - including showing non-printing characters.

...R

Robin2:

while (Serial.available() > 0) {

recvWithStartEndMarkers();




That is all wrong. It is not how the function is intended to be used.

Give me an example of the reply that you are having trouble receiving - including showing non-printing characters.

...R

It seemed to me that recvWithStartEndMarkers() reads only one line (until the endMarker character). So, i assumed that calling it "while (Serial.available() > 0)", would read each line until the last (like pop from a stack). Οbviously that's not true...

I'm trying to catch something like that:

+CWLAP:(2,"SSID1",-89,"00:1d:19:48:38:2a",6,-37)\r\n
+CWLAP:(4,"SSID2",-92,"66:ab:25:71:35:31",6,-54)\r\n
+CWLAP:(2,"SSID3",-80,"14:60:80:dd:48:b8",9,-52)\r\n
+CWLAP:(4,"SSID4",-65,"2c:95:7f:49:99:34",11,-32)\r\n

you need to keep your 'serial data' collection all inside that one loop, ONCE serial data is available.

Why are you jumping out to another function?

hanton:
i assumed that calling it "while (Serial.available() > 0)", would read each line until the last (like pop from a stack). Οbviously that's not true...

No. You should think of Serial.available() checking for single characters because the Arduino is fast and data arrives slowly .

I'm trying to catch something like that:

+CWLAP:(2,"SSID1",-89,"00:1d:19:48:38:2a",6,-37)\r\n

+CWLAP:(4,"SSID2",-92,"66:ab:25:71:35:31",6,-54)\r\n
+CWLAP:(2,"SSID3",-80,"14:60:80:dd:48:b8",9,-52)\r\n
+CWLAP:(4,"SSID4",-65,"2c:95:7f:49:99:34",11,-32)\r\n

If you don't know in advance how many lines of data will arrive I think you will need to develop some code that saves the value of millis() when a line arrives and newData == true.

Then you can check how long has elapsed since the last line was received and it if is long enough you will know that no more stuff is coming.

Those lines above seem to have about 50 characters so at 9600 baud each line would take about 50 millisecs to arrive. I reckon if there is a gap of 300 msecs there would probably be no more lines to come.

Hope this makes sense.

Of course if you do know in advance how many lines will arrive all you need to do is increment a counter when each line arrives.

...R

The data is followed by an empty line and a line containing "OK". AFAIR

AT+CWLAP

+CWLAP:(4,"WLAN-BC0543CECE45",-88,"bc:05:43:ce:ce:45",1,13)
+CWLAP:(3,"compair",-93,"c6:25:06:bf:3f:93",6,-16)
+CWLAP:(4,"Pascal24",-44,"f8:1a:67:a5:d6:af",6,-11)
+CWLAP:(3,"Kabel Deutschland",-92,"00:1e:58:0e:76:f4",10,-19)

OK

For an ESP-12F with

AT+GMR

AT version:0.40.0.0(Aug  8 2015 14:45:58)
SDK version:1.3.0
Ai-Thinker Technology Co.,Ltd.
Build:1.3.0.2 Sep 11 2015 11:48:04
OK

Whandall:
The data is followed by an empty line and a line containing "OK".

That would greatly simplify things - but it was not mentioned in Reply #14 even though I asked for everything in Reply #13

...R

It could depend on the AT software version of the ESP.
But I think the format was always like above when I tested the different modules.