next topic »

[on:]

I’m playing with a GPS module (https://www.sparkfun.com/products/8975) that communicates at 57600 bps (I think this refers to bits per second, NOT bytes per second). Dividing by 8 gives 7200 bytes per second, and assuming a typical NMEA sentence of up to 85 characters, this means a full sentence can arrive in about 12 milliseconds. (please correct my maths if I’ve got this wrong)

The serial input buffer on the arduino Mega is 64 bytes (or 64 characters) so if it isn’t checked very frequently, there’s a danger that the buffer will fill and NMEA data will be lost in between read()s.

The GPS library that I’ve got works fine in all low level testing when the read()s are taking place every few milliseconds and the data can be processed quickly. I know that the data’s good because I’m able to validate the NMEA checksum and confirm that more than 99% percent of the NMEA records are coming across correctly.

This library then gets used in a much bigger project where the arduino is also busy doing things like course correction, altitude calculation, logging and sending data. The result on the GPS library is that
•   The port the gps module is connected to only gets read once every 20-50 milliseconds.
•   The bytes awaiting processing each time a read occurs is between 80-100% of the buffer size (64 bytes).
•   The percentage of NMEA sentences successfully validated drops from about 99% to 70% or less.

What’s the solution? Would lowering the GPS bit rate work (if so, how is this done? I can’t spot a way to set this on the datasheet), would it help if I increased the size of the buffer that I hold the NMEA data in before I attempt to process it (this is currently set to 120 bytes). Or perhaps I should create a lower level library that just reads the GPS data from the input port into a buffer, and then only processes that data when I actually need it?

I've not posted code because I'm not looking for a specific fix, more some general ideas on what to do.

Thanks

[Reply #1 on:]

that communicates at 57600 bps (I think this refers to bits per second, NOT bytes per second). Dividing by 8 gives 7200 bytes per second,

Including the start and stop bit you have 10 bits so it is 5760 bytes per second.

The serial input buffer on the arduino Mega is 64 bytes (or 64 characters) so if it isn’t checked very frequently, ...

You aren't planning to check it at least every 10 mS?

If you can't change the rest you could increase the serial buffer size, but you might get to the point where you just aren't processing it fast enough.

and assuming a typical NMEA sentence of up to 85 characters, this means a full sentence can arrive in about 12 milliseconds.

1 / 5760 * 85 = 14 mS by my calculations.

Would lowering the GPS bit rate work?

You could just ignore the sentences that don't arrive completely. Wouldn't that amount to the same thing?

[Reply #2 on:]

The very product page you're referring to has the answer to your question.
Scroll down to "Great configuration software: Mini GPS" i guess this is some configuration software download (just guessing here, didn't download it).
Or scroll 1 line lower to "Example Configuration from DIYDrones".
Follow the links there, and you'll be able to change those settings you're asking for.

When ready setting these things, tell others what you found and if it does what you expected it to do.
You'll help other visitors to the Arduino forum by doing that.

[Reply #3 on:]

You aren't planning to check it at least every 10 mS?

I am planning to! But when the processing get's hectic, it often only gets read every 30 ms or longer.

you could increase the serial buffer size

How do i do this? I though the input serial buffer size on a mega was fixed to 64 bytes? Once the data arrives I copy it into my own buffer (whose size i can increase) until a complete record has arrived before processing it.

You could just ignore the sentences that don't arrive completely

that's what's currently happening, the problem is that I still need to process the record(i.e. validate the checksum) just to confirm if it's complete or not, also I sometimes get a whole string of NMEA records that aren't complete so I can be flying blind for 8 seconds or more without valid data!

[Reply #4 on:]

Do you need all sentences ?
If you don't you can check the sentence identifier, and if you're not interested in it, ignore the rest of it.

If you receive incomplete sentence, then you have to assume it is invalid (because you have no way of confirming it's validity).

[Reply #5 on:]

Follow the links there, and you'll be able to change those settings you're asking for.

Good link, but i didn't see anything to change the bit rate, but if I could switch off some of the record types I'm not interested in this would be an advantage.

If you receive incomplete sentence, then you have to assume it is invalid

True, but sometimes it's a couple of characters from the middle of the sentence that are missing, so I still need to verify against the NMEA checksum to confirm if it's valid or not (assuming the NMEA checksum isn't among the lost data)

Cheers

[Reply #6 on:]

Do you need all sentences ?
If you don't you can check the sentence identifier, and if you're not interested in it, ignore the rest of it.

If you receive incomplete sentence, then you have to assume it is invalid (because you have no way of confirming it's validity).

agreed
"GPRMC" sentence is enough for most applications
I don't use THIS gps, but in my case, I need only RMC an VTG , then, I just read these, and don't even check the checksum (but for my project, there is no "danger" ) .... I never get incomplete sentences .

PS: I had a quick look at the "miniGPS" software, it doesn't let you change the communication rate (at least, I didn't see the option)

[Reply #7 on:]

At the moment I'm testing for data at the port using the available() function. Is there any interupt that I can use to tell me when some data has arrived? If there was, I might be able to have a routine that as soon as more than say 30 bytes arrived I could copy the data to another bigger buffer and process it later when nothing else was happening.

cheers

[Reply #8 on:]

How do i do this? I though the input serial buffer size on a mega was fixed to 64 bytes?

The buffer is a software construct, so you can make it any size that you like, within the limits of the memory requirements of yor project.

[Reply #9 on:]

Did you read the 2nd link ?
First thing they do, is setting the baudrate from 4800  to 38400 by sending:

Code:

$PMTK251,38400*27

Of course if you want to set 4800 you'll have to calculate the correct checksum.

[Reply #10 on:]

The buffer is a software construct, so you can make it any size that you like

How do I change the size? what's the command to use?

The specification for the Mega suggests these are hardware ports, so I assumed the buffer size was fixed.

4 UARTs (hardware serial ports)

Did you read the 2nd link ?

not as thoroughly as you did !

Thanks

[Reply #11 on:]

At the moment I'm testing for data at the port using the available() function. Is there any interupt that I can use to tell me when some data has arrived? If there was, I might be able to have a routine that as soon as more than say 30 bytes arrived I could copy the data to another bigger buffer and process it later when nothing else was happening.

cheers

what I have done (probably not the best way, but it works well) :

I have a "readGPS" function with a while(Serial.available()) {.......}
wich reads each character, until it is a "13" , or until the number of char read is 159 (just to be sure I don't stay in the while loop and I don't write after the last byte of the string ) ,
and put them in a string (which is 160 byte long ) .
I call that function in the loop() , until I get the string I need (which begin with GPRMC or GPGTV)

It exists a interruption on serial event, but it didn't match my needs  
edit : here http://arduino.cc/en/Tutorial/SerialEvent

[Reply #12 on:]

The specification for the Mega suggests these are hardware ports, so I assumed the buffer size was fixed.

The UART is a hardware item, but the buffer is a software construct - the UART interrupts the MCU when a new character arrives, and the software put the character in a circular buffer.
Size is specified in {installation root}hardware/arduino/cores/arduino/HardWareSerial.cpp

[Reply #13 on:]

How do I change the size? what's the command to use?

The specification for the Mega suggests these are hardware ports, so I assumed the buffer size was fixed.

The hardware buffer size is two, so it can be reading the next byte while the previous one is buffered by an ISR.

Code:

// Define constants and variables for buffering incoming serial data.  We're
// using a ring buffer (I think), in which head is the index of the location
// to which to write the next incoming character and tail is the index of the
// location from which to read.
#if (RAMEND < 1000)
  #define SERIAL_BUFFER_SIZE 16
#else
  #define SERIAL_BUFFER_SIZE 64
#endif

See that 64? Change that.

[Reply #14 on:]

What GPS library are you using? I've used the tinyGPS library that needs to constantly read in from serial port and interpret. Regardless of the baud rate, your GPS is probably sending you sentences at a fixed interval, say once a second. I think a larger serial buffer WILL solve your problem as long as you can tend to it once a second or so, if you read spec sheet page 3 about default rate being 1Hz. You only need to capture n*85 bytes if you can only tend to the buffer every n seconds.

What routines are you running besides gps? Which is taking so much time? Can it be turned into a state machine to run one bit at a time?