Reading Serial reliably

I have a GSM / GPRS Shield connected to an Arduino Duemilanove, and i have much trouble communicating to it reliably.
I set the Shield so that it takes commands via SoftwareSerial and then spits out any response to that Command via regular Serial - this has improved my issues a fair bit. Whats happening is that i send a command to the GSM Shield, and i get a reply of unknown length back. Or often i don't. The below process is fairly consistent and demonstrates my issue with an example:

  1. Reset Arduino & open Serial Console
  2. Send a new SMS to the GSM shield via Mobilephone
  3. Run check_sms command to list new SMS
  4. The output in the console is either not existing or cut off somewhere.
  5. Run check_sms command to list new SMS again
  6. The output is, in 95% of cases, what it should be.
  7. Send another new SMS via Mobilephone
  8. Rinse & repeat. Output is wrong the first time, it's nearly always correct the second or third time.

So why is it that i need to run a command twice? I played around with different methods, timings and delays for days now, and just cant get it to work reliably. When i started out using pure SoftwareSerial to communicate back and forth with the GSM Shield it was even worse, and get a lot of garbled output. Now the issue is that i don't reliably get all the output, or sometimes even no output.

I would be most grateful for someone to tell me just what the right way(TM) is to communicate reliably with my GSM Shield, or what the problems are in my Code or with my Timing. How to solidly read Data from the Device so that i can nearly guaranteed get the full output? I guess it all has to do with how fast the Shield processes incoming commands and how long it takes to send back a reply.

Thanks a lot.

#include <SoftwareSerial.h>

SoftwareSerial cell(2,3);

char ch;

String command = "";

void setup(){
	Serial.begin(9600);
	cell.begin(9600);
	Serial.println("Starting ATWIN Communication...");
}

void loop(){
	if(Serial.available()){
		command = get_serial_contents();
	}

	if(command == "check_sms")
		check_sms();

	command = "";
}

void check_sms(){
	Serial.println("Checking SMS");
	cell.println("AT+CMGL=\"ALL\"");
	delay(100);

	while(!Serial.available() >0){};

	if(Serial.available()>0){
		while(Serial.available()>0){
			delay(1);
			ch = Serial.read();
			Serial.print(ch);
		}
	}
}

String get_serial_contents(){
	Serial.println("Getting Serial data");
	char character;
	String contents;

	while(Serial.available()){
		delay(10);
		character = Serial.read();
		contents.concat(character);
	}
	Serial.flush();
	return contents;
}
String command = "";

There's your problem, right there. Ditch the String class.

	while(Serial.available()){
		delay(10);
		character = Serial.read();
		contents.concat(character);
	}
	Serial.flush();

While there is a byte in the buffer to read, dawdle around for a while, then read that character. As soon as there is nothing in the buffer any more, delete everything that is in the buffer without reading it (pre-1.0) or block waiting for unsent data to be sent (1.0+). Now, explain why the dawdling and what possible useful function the flush() function is performing.

Since you are going to store the returned String in a global variable, anyway, why make a local variable that needs to be copied and then freed?

Hi there,

thank you for your comments. String seemed a simple way to read in a command such as "check_sms" or "do_this" and then act upon it.

I have removed all references to String now in my Example, and also removed the useless flushing and some of the dawdling. Some of it may be because i was trying a lot of different things to see what makes a difference.

With the below Code, the problem is still the same; first time running "check_sms" after resetting / repowering gets me only parts of the output. Subsequent runs of the same command, gives me the output as expected. :frowning:

#include <SoftwareSerial.h>

SoftwareSerial cell(2,3);

char command;

void setup(){
  Serial.begin(9600);
  cell.begin(9600);
  Serial.println("Starting ATWIN Communication...");
}

void loop(){
  if(Serial.available()){
    command=Serial.read();
    if(command == 'c'){
     check_sms();
     command=0;
    } 
  }
}

void check_sms(){
  char ch;
  Serial.println("Checking SMS");
  cell.println("AT+CMGL=\"ALL\"");
  delay(100);

  while(!Serial.available() >0){};
  
    if(Serial.available()>0){
      while(Serial.available()>0){
        delay(1);
        ch = Serial.read();
        Serial.print(ch);
      }
    }
}

Every SMS has an explicit end-of-data marker. You need to figure out what that is, and read until that marker arrives. Assuming that the response will arrive in some fixed period of time will NEVER work.

The end of record marker may not be a single character. It may be a series of characters, like two carriage returns in a row, or even more, like or in order.

So this Code below will probably get me scolded for being wrong, but what i was trying to archive is to simply read on the Serial for 5 Seconds so to give the Shield enough time to send stuff. Its the same problem - first time does not product much output, second and subsequent times work fine.

I could look for an EOD Marker, which in this Case should be "OK\n", but the problem is that the first time i run my command, that particular OK\n Sequence never ever comes, no matter how long i wait. I tried to look for EOD markers before as that really makes most sense, but in my case the code will just enter an endless loop as sometimes there just simply is no specified EOD marker.

Thank you for more guidance

#include <SoftwareSerial.h>

SoftwareSerial cell(2,3);

char command;

void setup(){
  Serial.begin(9600);
  cell.begin(9600);
  Serial.println("Starting ATWIN Communication...");
}

void loop(){
  if(Serial.available()){
    command=Serial.read();
    if(command == 'c'){
     check_sms();
     command=0;
    }
  }
}

void check_sms(){
  char ch;
  Serial.println("Checking SMS");
  cell.println("AT+CMGL=\"ALL\"");
  delay(100);

  while(!Serial.available() >0){};

  long int start = millis();
  while((millis() - start) < 5000){
    if(Serial.available()>0){
      delay(1);
      ch = Serial.read();
      Serial.print(ch);
    }
  }
  
  Serial.println("End of Data");
}
    if(Serial.available()>0){
      delay(1);
      ch = Serial.read();
      Serial.print(ch);
    }

Here's the part I have a problem with. You know that there is a character available to be read. Maybe even more than one. What are you waiting for? Why is the delay() there at all?

but the problem is that the first time i run my command, that particular OK\n Sequence never ever comes, no matter how long i wait.

Does this problem recur of there is a long wait between SMS's or between commands to get messages from the phone? If not, then simply send the command in setup to wake the phone up and get an incomplete reply that you discard.

In various methods that i tried adding the delay(1) to my while loop seemed to help getting a more sensible result from my serial output. In different code snippets i found by researching online and also in a Arduino Getting Started Book thingy, there is something that reads like this:

while(Serial.available()>0){
delay(1) // let the buffer fill up
int numChar = Serial.read();
// ...
}

"Let the buffer fill up." It seemed to make sense to have a few characters in the buffer before starting to read on it, so not to get into a situation where one is trying to read the next character before it has arrived. That's where it came from - i realize it may be useless after an if statement. I was troubleshooting and just trying lots of different stuff.

As for the Phone waking up: I have added the following to my setup() function, which seems to have improved things considerably:

cell.println("AT");
delay(50);
if(Serial.available()>0){
while(Serial.available()>0){
  delay(1);
  ch = Serial.read();
}
}

I re-powered the Unit, waited a few seconds before running my first check_sms, and voila! The output seems fine which is better than before. I will do more testing now and see if this has solved it - if it did i am a super happy camper...

I don't know whether this applies to the serial connection between the Arduino and your GSM/GPRS shield, but it's quite common for async serial links to take a character or two to get into synch with the bitstream when the link initially comes up. This could affect the serial stream in both directions. In that case it'd help to have some handshake between them at startup just to let them both get the serial stream synched.

If your code is correct, it should not be necessary to have any delays in the code handling serial port input.