Parsing Cell SMS text messages

Can anyone recommend an algortihm or library that will parse SMS messages?

I am using the NewSoftSrial library with inconsistent results. I am not sure if I am just not clear on the proper way to retrieve data from NSS or if its something else.

Receiving data, and parsing it, are two completely different tasks. If you are having trouble receiving the data, having a functioning parser that you pass garbage to will not be very useful.

How about you post your code, and explain what "inconsistent results" means?

//##### Include our libraries #####

#include <NewSoftSerial.h>



//##### Constants #####
#define BUFFSIZE 512




//##### Global Vars #####
char szATBuff[BUFFSIZE];
char szInPut = 0;
NewSoftSerial cell(2,3); 
int CELL_REG = 0;
int CELL_AT = 0;
int CELL_SMSMODE = 0;
int nBufPos;
int msg = 0;










void setup() {
  
  Serial.begin(9600);
  cell.begin(9600);
  
  
  
   
  
  
  delay(1500);
 
 
}


  
  


void loop() {
      
    while(CELL_REG == 0 || CELL_AT == 0) {  
        ATGetResp(0);
        ParseATResp();
    }
        
        
        ATGetSMS();
        
        
        
            
    
    
}
  


//************************************************************
// Get AT command respnse from the Cell module. Starts and
// ends with CR LF.
//************************************************************
void ATGetResp(int flag) {
    
    char c;
    memset(szATBuff, '\0', BUFFSIZE);
    Serial.println("Cleared Buff!");
    Serial.println(szATBuff);
    delay(3000);
    nBufPos = 0; // start at begninning
    int nCnt = 0;
    
    
    while (cell.available() > 0 && nCnt < BUFFSIZE) {

      c = cell.read();
      
      if (c == '\r') {
          szATBuff[nBufPos] = c;
          nBufPos++;
          c = cell.read();
      
          if (c == '\n') {
            szATBuff[nBufPos] = c;
            nBufPos++;
            szATBuff[nBufPos] = '\0';
           
            return;
          }
      }
      
      if (c == '\n'){
         
         continue;
      }
       
      szATBuff[nBufPos] = c;
      nBufPos++;
      
      
      nCnt++;
    }
        
      szATBuff[nBufPos] = '\0';
}

  

//************************************************************
// This will be used to recv data via SMS
//************************************************************
void ATGetSMS(void) {
  
        if (CELL_REG == 1 && CELL_AT == 1) {
          
            if (!CELL_SMSMODE) {
              cell.println("AT+CMGF=1");
              CELL_SMSMODE = 1;
              Serial.println("CMFG SET to TEXT");
              ATGetResp(1);
              ParseATResp();
              
            }
            
            
            
           
                //cell.flush();
                cell.println("AT+CMGL=\"ALL\"");
                
                ATGetResp(1);

                Serial.println("DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println(szATBuff);
                Serial.println("DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                //cell.flush();
                
                
               
            
        }
       

}






//************************************************************
// Parse AT Response string
//************************************************************
void ParseATResp(void) {
 
        
      
        
        if (strstr(szATBuff, "+SIND: 11") != 0) {
            CELL_REG = 1;
            return;
        } 
        
        
        if (strstr(szATBuff, "+SIND: 4") != 0) {
            CELL_AT = 1;
            return;
        }
        
        
        i
}

I am trying to read SMS records from my cell module. But it seems like I am rereading part of the previous data. Like the buffer is filling up and is circular so it starts overwriting the old data. so when I issue a second read I see some of the new record and some old record on the end.

I will try to post my results when I get to where I can test.

If the data of importance is terminated by \n\r or \r\n, there seems to be no reason to store either \r or \n in the szATBuff array. (By the way, the Arduino isn't running a Windoze OS, so lose the stupid sz on the front of szATBuff. It's obvious from where/how it is used that it is a string. Only morons at Microsoft can't remeber stuff like that.)

You have two values in the function that count how many bytes are written to the array. They are not updated together consistently. Perhaps that is a source of the problem. Try making do with just one.

Finally, you are presuming that all the serial data has arrived before you begin reading any. If ATGetResp is supposed to extract an entire packet from the serial buffer, it should not return until both the \n and \r have been received (and discarded), regardless of how long that might take. As it is, the function returns when the serial buffer is empty, even if the end-of-packet marker has not been received. Parsing an incomplete packet will not be possible (or useful), should you eventually add code to do that.

A little bit OT:

is terminated by \n\r or \r\n

It is ALWAYS \r\n The rationale behind this comes from the mechanical TTY machines ;)

I know that they are both present, but I can never remember the correct order. Oh, well.

jee sorry about the sz stuff. Took it out.

Here is the new code…

//##### Include our libraries #####
#include <NewSoftSerial.h>



//##### Constants #####
#define BUFFSIZE 512




//##### Global Vars #####
char ATBuff[BUFFSIZE];
char szInPut = 0;
NewSoftSerial cell(2,3); 
int CELL_REG = 0;
int CELL_AT = 0;
int CELL_SMSMODE = 0;
int nBufPos;
int msg = 0;










void setup() {
  
  Serial.begin(9600);
  cell.begin(9600);
 
  
  delay(1500);
 
 
}


  
  


void loop() {
      
    while(CELL_REG == 0 || CELL_AT == 0) {  
        ATGetResp();
        ParseATResp();
    }
        
        
        ATGetSMS();
        
        
        
            
    
    
}


//************************************************************
// Get AT command respnse from the Cell module. Starts and
// ends with CR LF.
//************************************************************
void ATGetResp() {
    
    char c;
    memset(ATBuff, '\0', BUFFSIZE);
    
    nBufPos = 0; // start at begninning
    int nBytes = 0;
    
    nBytes = cell.available();
    
    if (nBytes > 0) {
      
      for (int iii = 1; iii <= nBytes; iii++) {
        
        if (nBufPos == BUFFSIZE - 3)
          break;
          
        c = cell.read();
               
        if (c == '\r') {
            ATBuff[nBufPos] = c;
            
            c = cell.read();
        
            if (c == '\n') {
 
              ATBuff[nBufPos] = '\0';
             
              return;
            }
        }
        
        
         
        ATBuff[nBufPos] = c;
        nBufPos++;
      }
     
    }
        
      ATBuff[nBufPos] = '\0';
      return;
}

  






//************************************************************
// This will be used to recv data via SMS
//************************************************************
void ATGetSMS(void) {
  
        if (CELL_REG == 1 && CELL_AT == 1) {
          
            if (!CELL_SMSMODE) {
              cell.println("AT+CMGF=1");
              CELL_SMSMODE = 1;
              Serial.println("CMFG SET to TEXT");
              ATGetResp();
              ParseATResp();
              
            }
            
            
            
           
                //cell.flush();
                cell.println("AT+CMGL=\"ALL\"");
                delay(3000);
                //int n = cell.cpybuf(ATBuff, BUFFSIZE);
                ATGetResp();
                
                     
                
                
                //Serial.println(n);
                Serial.println(cell.overflow());
                Serial.println();
                Serial.println();
                Serial.println("HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println(ATBuff);
                Serial.println("HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println();
                Serial.println();
                ATGetResp();
                Serial.println("DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println(ATBuff);
                Serial.println("DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println();
                Serial.println();
                ATGetResp();
                Serial.println(cell.overflow());
                Serial.println();
                Serial.println();
                Serial.println("HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println(ATBuff);
                Serial.println("HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println();
                Serial.println();
                ATGetResp();
                Serial.println("DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println(ATBuff);
                Serial.println("DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println();
                Serial.println();
                ATGetResp();
                Serial.println(cell.overflow());
                Serial.println();
                Serial.println();
                Serial.println("HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println(ATBuff);
                Serial.println("HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println();
                Serial.println();
                ATGetResp();
                Serial.println("DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println(ATBuff);
                Serial.println("DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
                Serial.println();
                Serial.println();
                
                
                delay(3000);
               
            
        }
       

}






//************************************************************
// Parse AT Response string
//************************************************************
void ParseATResp(void) {
        
        if (strstr(ATBuff, "+SIND: 11") != 0) {
            CELL_REG = 1;
          
            return;
        } 
        
        
        if (strstr(ATBuff, "+SIND: 4") != 0) {
            CELL_AT = 1;
           
            return;
        }

}

This is what I am seeing:

Cell AT Ready

CMFG SET to TEXT

1

HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

OK

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

0

(first record)
HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

+CMGL: 1,0,“REC READ”,"+17703173606",“10/09/24,20:02:00+00”

HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

gggggcccvvbbb

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

0

HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

+CMGL: 2,0,“REC READ”,"+17703173606",“10/09/24,20:04:00+00”

HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

gvvvbjoihiihij

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

1

HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

+CMGL: 3,0,“REC READ”,"+17703173606",“10/09/26,18:00:05+00”

HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

aaaaaaaaa

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

0

(record 4-- bad)
HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

+CMGL: 4,0,“REC READ”,"++CMGL: 1,0,“REC READ”,"+17703173606",“10/09/24,20:02:00+00”

HEADER>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

gggggcccvvbbb

DATA>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

in the ATGetSMS I call the ATGetResp multiple times before looping back round and re-issuing the command to get SMS msgs. As you can I get the header then data in order and all looks good for the first 3 SMS records. Then on the 4th record I have a problem… Record all the sudden overwrites the tail of record 4 and the data portion is that of record 1…

I am not 100 percent for sure but I think whats happening on the 4th record is that NSS is hitting the end of the buffer then wrapping around and the rest of the 4 th record is being lost/dropped. And what I am seeing here is the result of a ring buffer that is getting full and wrapping. But I dont really know how to handle this. I have tried to issue the SMS READ command then read only the first record then delete it and do NSS flush then issue the READ cmd again and get the response from I feel should be a 0 out buffer... so the first record now be should whatever is next in the cell module since I deleted the first record... should this work?

How do I send text message to my iphone from the arduino. I have an arduino with a gsm playground board and i only need to be able to send a text message to my iphone. how to i program the arduino to do this? is there a library with example code like this, or has anyone done something similar to this before?

jdrogers are right 4th message overflow from your buffer so that why you get +CMGL: 4,0,"REC READ","++CMGL: 1,0,"REC READ",".... +CMGL: 4,0,"REC READ","+ is from your 4 message then your buffer overflow and your reader start from 0. Basically you can do like jdrogers write read first message and clear buffer OR you can do ring buffer and handle overflow by continuing buffer write from 0 if (buffer position >= BUFFSIZE).

Your code

if (nBufPos == BUFFSIZE - 3)
          break;

Proposed ring buffer

if (nBufPos == BUFFSIZE - 3) 
{
  nBufPos = 0;
}

Proposed code work if your Arduino run and poll received SMS when you send SMS message (it need changes to reader also), but if you want send 4 message and then start read those from GSM only solution is make bigger buffer. Anyway with

if (nBufPos == BUFFSIZE - 3)
          break;

You can receive only four first message not anymore.

Anyway THX for example and idea I will use it (with ring buffer) in my GSM remote / alarm system.