Pages: [1]   Go Down
Author Topic: Serial bufer size  (Read 463 times)
0 Members and 1 Guest are viewing this topic.
Portugal
Offline Offline
Edison Member
*
Karma: 37
Posts: 1531
Pretending you know everything then you will learn nothing.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello all
I'm having a problem which I believe it's a serial buffer overflow problem.I'm working with a modem GSM and to receive a message I seend from the Serial port this command:
Code:
AT+CMGR=1
Then the modem replies back to me the message in position 1 of is memory.
To read all that data I create this function:
Code:
void extract_SMS_Data()
{
Serial.print("AT+CMGR=1");//Send instrution to read the SMS
delay(30);
Serial.print(0x0d,BYTE);//Makes an Enter @ the end
int i=0;
while (Serial.available() > 0)
  {
    sms_array[i] = Serial.read();//char sms_array is declared global
    i++;
  }
I was expecting to all bytes get stored in sms_array that was declared with a 120 elements
After that if I loop all the elements of that array I just have then correctly until position  sms_array[72]
After that I get nothing
I'm reusing some of this code from an old project.
At arduino docs I found that the Serial buffer has 64 bytes, so I perhaps my code is not reading the bytes in Serial buffer fast  enough
Does the Serial buffer size change in arduino IDE 1.0.3?
Like I said I have part of this code working in lower versions
« Last Edit: February 18, 2013, 05:31:26 pm by HugoPT » Logged

Debian,Mint,Ubuntu
Arduino Mega 2560
Arduino Nano
Arduino Duemilanove
MAC OS Montain Lion
Raspberry PI Model B


Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 601
Posts: 48543
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
I was expecting to all bytes get stored in sms_array that was declared with a 120 elements
That hardly seems a realistic expectation, given that serial data transmission is ssslllooowww. The Arduino is orders of magnitude faster than serial data can be transmitted.

Is there some "Hey, this is the end of the message!" marker in the message response? If so, you need to wait for that to arrive, in a while loop.

Quote
At arduino docs I found that the Serial buffer has 64 bytes, so I perhaps my code is not reading the bytes in Serial buffer fast  enough
It does, and you are. You are emptying it faster than it is being filled, as a matter of fact.

Quote
Does the Serial buffer size change in arduino IDE 1.0.3?
No, it does not.

Working in what version?
Logged

0
Offline Offline
Shannon Member
****
Karma: 199
Posts: 11639
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Your loop will drop out if it runs ahead of the incoming serial stream.  Perhaps you need to incrementally populate the array
in loop() and have some other function call regularly check to see when enough input has arrived.

Alternatively if you know how many bytes are expected, keep reading till that many have arrived (although that will block the
rest of the program).

You current code is presumably consuming all the bytes currently in the buffer, then available() returns zero, and it simply
drops out (when it should wait for more bytes coming in until all the expected ones have arrived).

You don't check for buffer overrun - you'll need to fix that or the program can crash.
Logged

[ I won't respond to messages, use the forum please ]

Portugal
Offline Offline
Edison Member
*
Karma: 37
Posts: 1531
Pretending you know everything then you will learn nothing.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
You don't check for buffer overrun - you'll need to fix that or the program can crash.
How can I check that? is there some bit registry that I can "look" to see if an overflow happens?
Code:
Working in what version?
It was working on version 23
Code:
Is there some "Hey, this is the end of the message!" marker in the message response? If so, you need to wait for that to arrive, in a while loop.
The size of the message seend by the modem varies, it depend how much the body of the message contains.
Since the message is big, because I also receive some wast information like sender id , time of the message, date and finally the body of the message which is the only part i'm interested and the sender id.So I have no way to filter it i belive.
Quote
You are emptying it faster than it is being filled, as a matter of fact.
If I put a small delay here does it will prevent this?
Code:
 
int k=0;
  while (Serial.available() > 0)
    {
      sms_array[k] = Serial.read();
      delay(30);//To prevent reading faster than Serial data get's in
      k++;
    }
Logged

Debian,Mint,Ubuntu
Arduino Mega 2560
Arduino Nano
Arduino Duemilanove
MAC OS Montain Lion
Raspberry PI Model B


Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 601
Posts: 48543
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
How can I check that? is there some bit registry that I can "look" to see if an overflow happens?
No. You've declared an array of a fixed size. You have an index into that array. Check that the value of the index is small enough so that whatever gets written WILL fit.

Quote
It was working on version 23
Lots of changes from 0023 to 1.0+. Most notably, the buffer is half the size, and outgoing data is also buffered.

Quote
If I put a small delay here does it will prevent this?
Sure. Try two weeks. A delay() is a band-aid. Figure out what the end-of-message marker is, and wait until that arrives. If there is no defined end of message marker, get a hammer and then a new device that is a little more intelligent.

Logged

New Jersey
Offline Offline
Faraday Member
**
Karma: 65
Posts: 3638
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
The size of the message seend by the modem varies, it depend how much the body of the message contains.
Since the message is big, because I also receive some wast information like sender id , time of the message, date and finally the body of the message which is the only part i'm interested and the sender id.

What do these messages look like? How do you know that the message body is done and you're now reading the sender id? Can you use that knowledge to figure out how to parse the message?
Logged

Portugal
Offline Offline
Edison Member
*
Karma: 37
Posts: 1531
Pretending you know everything then you will learn nothing.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

 
Quote
If there is no defined end of message marker, get a hammer and then a new device that is a little more intelligent.
smiley-grin Very funny   smiley-grin smiley-grin smiley-grin smiley-lol
Code:
Figure out what the end-of-message marker is, and wait until that arrives
But doing this isn't sure that I will overflow the buffer?
Lets say I found the marker and it is on 80 position byte (message is long) , I had overflow it along time ago then and never get it done right?
My idea was when I get something I put it in a incoming array to prevent Serial buffer overflow, then I go on each element and extract it to another parts, getting that way the sender ID and the message.
In my code I was not detecting the end of the message in fact, I just collect the 100 bytes and its done.I assume the body of the message never gets there, so this was my solution since I dont have a marker.I know it's stupid solution but it does the job.
Here is an example of the intire message the modem gives back:
Code:
+CMGR: "REC UNREAD","+35196XXXXXXXX",,"13/02/18,23:07:09+00"
This is the Body
« Last Edit: February 18, 2013, 06:28:44 pm by HugoPT » Logged

Debian,Mint,Ubuntu
Arduino Mega 2560
Arduino Nano
Arduino Duemilanove
MAC OS Montain Lion
Raspberry PI Model B


Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 601
Posts: 48543
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
But doing this isn't sure that I will overflow the buffer?
SMSs are typically limited to 140 characters. Size the receive buffer appropriately.

Quote
Lets say I found the marker and it is on 80 position byte (message is long) , I had overflow it along time ago then and never get it done right?
Depends on the size of the receive buffer. If it's 40 bytes, and the end of record marker is at position 80, then the message won't fit in the buffer. If the buffer is 140 characters, 80 bytes should have ample room.

Quote
My idea was when I get something I put it in a incoming array to prevent Serial buffer overflow, then I go on each element and extract it to another parts, getting that way the sender ID and the message.
That's a mighty fine idea.

Quote
I know it's stupid solution but it does the job.
It hasn't failed yet, you mean. Receive a long enough text message and it will, if you assume that long enough is less than 140 characters and don't test for overflow.

Quote
Here is an example of the intire message the modem gives back:
I routinely get multipart messages, because the text message is too long to fit in one message. Your mileage will vary.

Printing each and every character in that message in HEX format just might give you a clue what the end of message marker is.
Logged

Left Coast, CA (USA)
Offline Offline
Brattain Member
*****
Karma: 361
Posts: 17259
Measurement changes behavior
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Well technically I don't think the serial library will allow or cause a buffer overflow condition, just a buffer wrap around condition, so the damage will be limited to becoming an invalid data stream. With proper protocol such as good message delimiters your code should be able to recover from such situations?

Lefty
« Last Edit: February 18, 2013, 06:46:40 pm by retrolefty » Logged

Global Moderator
Offline Offline
Brattain Member
*****
Karma: 473
Posts: 18695
Lua rocks!
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@OP: http://www.gammon.com.au/serial
Logged

New Jersey
Offline Offline
Faraday Member
**
Karma: 65
Posts: 3638
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Your example message looks like the body is separated from the header by CRLF or some perhaps just LF. You could just read and throw away characters until you see a LF, then copy the body to your buffer. Does the body have a similar entry at the end of it?
Logged

Rome, Italy
Offline Offline
Sr. Member
****
Karma: 20
Posts: 442
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Aren't you receiving a closing OK (or ERROR) in a line of its own?
Logged

Pages: [1]   Go Up
Jump to: