Go Down

Topic: GPS: set update frequency (Read 21004 times) previous topic - next topic

wildbill

Quote
So if I get you right, the occurrences of "next" in the middle of a sentence shows the characters that the gps managed to put into the buffer during the 50ms delay + loop processing time?

Yes.
Quote
The sender will wait if the receivers buffer is full?
Meaning, if the MPU is not reading the buffer fast enough, the gps will queue messages waiting for the MPU to free up buffer? Assuemed the GPS also has a buffer, this would totally explain why I see delayed data!

No. If the arduino is transmitting more data that it can buffer, it blocks until what's left will fit in the buffer. During that blocking, you may lose data from the GPS.
Quote
From this I think I learned that buffer behavior is to drop new data when full rather than saving the last 64 bytes received. Is there a way to flip this behavior so that when buffer is full, the oldest data is dropped, allowing new data to always be written?

Your understanding is correct. Changing it would require hacking the core libraries and even if you could, you would still have dropped data. Better to arrange it so that you process the serial data in a timely fashion.
Quote
Thats one possibility. Another could be to monitor the datastream, waiting for the start of the wanted message, then collect the full message and pass it to the encode().

Indeed.

petterg


No. If the arduino is transmitting more data that it can buffer, it blocks until what's left will fit in the buffer. During that blocking, you may lose data from the GPS.


Ah. So you were aiming towards the serial.print() to the computer could lock up the processing so that Serial1.read() from the gps would meanwhile run late?

I've been looking into the TinyGPS++ library now. To me it looks like only GGA and RMC messages are recognized. It also looks like encode() performs just the same amount of processing even when the message is not among those recognized. There is room for improvements.

petterg

I did not get any better doc from skylab, so I did some modifications to the library, making it not parse any messages if the first 5 letters indicate it's not among those two that the library supports. For a small set of sample data this improved processing time from 75ms to 48ms. (Testdata was feeded to the library directly from a variable, not from the gps.)

However, this did not solve the problem with delayed updates.

What I think happens is this:
Data is written from GPS to arduino serial buffer. Buffer is read, first message is parsed. While the first message is parsed the buffer goes full. Buffer is read, data is dropped, start of the second supported message is detected, but the second half of the second message is dropped because of buffer overflow. Hence end of message is not detected, so it keeps reading and parsing unsupported messages until an end of message is detected. Then it gets an checksum error and the message is dropped.
Over time this error builds up. The message containing position data is quite a bit longer than the one containing date/time data. Hence the message containing position data frequently suffers from buffer overflow and is dropped.

So whats the options to get around this? Some of the options I see is:
1) Increase the buffer size so that it will be able to contain one sample of each message sent from the gps. That would give the library 1 second figure what to drop or parse.
2) Change the buffer behavior so that it's dropping the oldest data when full rather than not receiving new data.
3) Write the code so that it's somehow synchronized with the gps. Then it would be able to empty the buffer just before the interesting messages arrives.
4) Get an arduino due, which would parse the data faster, hopefully fast enough to keep the buffer from overflowing

What do you think of these ideas?
Does anyone have other ideas?

PaulS

Quote
1) Increase the buffer size so that it will be able to contain one sample of each message sent from the gps. That would give the library 1 second figure what to drop or parse.

That might work, but it shouldn't take a whole second to parse the data that you got.

Quote
2) Change the buffer behavior so that it's dropping the oldest data when full rather than not receiving new data.

That's going to be a lot harder than you think.

Quote
3) Write the code so that it's somehow synchronized with the gps. Then it would be able to empty the buffer just before the interesting messages arrives.

Hmmm...

Quote
For a small set of sample data this improved processing time from 75ms to 48ms.

This suggests to me that you are not loosing data. New data shouldn't begin to arrive until the Arduino has had plenty of time to parse the last sentence collected.

Quote
4) Get an arduino due, which would parse the data faster, hopefully fast enough to keep the buffer from overflowing

Maybe. I haven't heard other people complaining about how long it takes to parse the data. And, the numbers you posted show it isn't taking you that long, either.
The art of getting good answers lies in asking good questions.

petterg


Quote
1) Increase the buffer size so that it will be able to contain one sample of each message sent from the gps. That would give the library 1 second figure what to drop or parse.

That might work, but it shouldn't take a whole second to parse the data that you got.

The gps seems to be sending all sentences continuously until sent, then not send anything for the rest of the 1 second data transfer interval. So a buffer that will hold 1 second of data will need to have the same size as on that holds 0.3 seconds of data.


Quote
2) Change the buffer behavior so that it's dropping the oldest data when full rather than not receiving new data.

That's going to be a lot harder than you think.

Probably, yes....


Quote
3) Write the code so that it's somehow synchronized with the gps. Then it would be able to empty the buffer just before the interesting messages arrives.

Hmmm...

And I think, even if the sync-part should turn out doable, I doubt it would solve the problem.


Quote
For a small set of sample data this improved processing time from 75ms to 48ms.

This suggests to me that you are not loosing data. New data shouldn't begin to arrive until the Arduino has had plenty of time to parse the last sentence collected.

Well, yes and no. The full set of sentences arrive in one bulk. The issue is that parsing is performed as the sentences arrive. Assume those 48ms is 70% parsing position, 25% parsing date/time and 5% deciding the arriving message is not supported. That would imply that reading the buffer halts for 34ms during one block of sentences while parsing position.  If buffer is 64byte there will be an overflow if the data rate is more than 1882 bytes/second. What is the data rate for 9600baud serial? 9600byte/sec? If so, the buffer surely will overflow.


Quote
4) Get an arduino due, which would parse the data faster, hopefully fast enough to keep the buffer from overflowing

Maybe. I haven't heard other people complaining about how long it takes to parse the data. And, the numbers you posted show it isn't taking you that long, either.

If the assumption above is correct, processing at 16MHz*9600/1882=81MHz would solve the problem - IF the code is somewhat in sync with the gps, so that the code will not be busy doing other stuff when a new set of sentences start. After all, the target for the code is primarily to make use of the gps data, not just do the readings.

doughboy

probably not the answer you are looking for, but just throwing this out as an option.

can you try another gps? neo6m now costs like $17.
can you try other firmware besides tinygps? I know tinygps can be used in quadcopter equipped with gps, but the i2c_gps_nav program that provides similar functionality is more stable and reliable imho.

PaulS

Quote
The gps seems to be sending all sentences continuously until sent, then not send anything for the rest of the 1 second data transfer interval.

Then, perhaps you need to clone the TinyGPS library so that you can copy the buffer that TinyGPS writes to when the end of the sentence that you want to parse arrives, and then, after the last sentence in the burst arrives, parse the copy. You should be done with the parsing by the time the next batch starts to arrive.
The art of getting good answers lies in asking good questions.

petterg


Quote
The gps seems to be sending all sentences continuously until sent, then not send anything for the rest of the 1 second data transfer interval.

Then, perhaps you need to clone the TinyGPS library so that you can copy the buffer that TinyGPS writes to when the end of the sentence that you want to parse arrives, and then, after the last sentence in the burst arrives, parse the copy. You should be done with the parsing by the time the next batch starts to arrive.


Thats very true.
Wouldn't that, in matter of resource usage, essentially be the same as increasing the buffer size? What is the disadvantages of increasing the buffer size?

petterg


can you try another gps? neo6m now costs like $17.

Why would that gps be better?


i2c_gps_nav

Thanks. I'll have a look at it.

PaulS

Quote
What is the disadvantages of increasing the buffer size?

In my opinion, that allows for sloppy coding. There are situations where it is unavoidable, I guess. The biggest drawback is that there limited memory available, and there is only one value that defines the size of the incoming AND outgoing serial buffers.

Still, it may be necessary, in your case, to make the change.
The art of getting good answers lies in asking good questions.

Docedison

#55
Oct 17, 2013, 03:49 am Last Edit: Oct 17, 2013, 03:51 am by Docedison Reason: 1
Some time back... (the first of the year..) I was using a Skylabs SKM53... I had no problems connecting in the house. I had an update once a second.
I thought this might be helpful..
Code: [Select]
 
 // uncomment this line to turn on RMC (recommended minimum) and GGA (fix data) including altitude
 GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCGGA);
 // uncomment this line to turn on only the "minimum recommended" data
 //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);
 // For parsing data, we don't suggest using anything but either RMC only or RMC+GGA since
 // the parser doesn't care about other sentences at this time
 
 // Set the update rate
 GPS.sendCommand(PMTK_SET_NMEA_UPDATE_1HZ);   // 1 Hz update rate
 // For the parsing code to work nicely and have time to sort thru the data, and
 // print it out we don't suggest using anything higher than 1 Hz

 // the nice thing about this code is you can have a timer0 interrupt go off
 // every 1 millisecond, and read data from the GPS for you. that makes the
 // loop code a heck of a lot easier!

This from the Adafruit GPS tutorial and sketch... Using Mikal Hart's TinyGPS code.
This was a "No Brainer" I just read all the documentation first... Something that seems to be missing here..    EDIT Doc

Doc
--> WA7EMS <--
"The solution of every problem is another problem." -Johann Wolfgang von Goethe
I do answer technical questions PM'd to me with whatever is in my clipboard

PaulS

Quote
I just read all the documentation first

All of what documentation? Where did you get the snippet you posted?
The art of getting good answers lies in asking good questions.

petterg


Some time back... (the first of the year..) I was using a Skylabs SKM53... I had no problems connecting in the house. I had an update once a second.
I thought this might be helpful..
Code: [Select]
 
 // uncomment this line to turn on RMC (recommended minimum) and GGA (fix data) including altitude
 GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCGGA);
 // uncomment this line to turn on only the "minimum recommended" data
 //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);
 // For parsing data, we don't suggest using anything but either RMC only or RMC+GGA since
 // the parser doesn't care about other sentences at this time
 
 // Set the update rate
 GPS.sendCommand(PMTK_SET_NMEA_UPDATE_1HZ);   // 1 Hz update rate
 // For the parsing code to work nicely and have time to sort thru the data, and
 // print it out we don't suggest using anything higher than 1 Hz

 // the nice thing about this code is you can have a timer0 interrupt go off
 // every 1 millisecond, and read data from the GPS for you. that makes the
 // loop code a heck of a lot easier!

This from the Adafruit GPS tutorial and sketch... Using Mikal Hart's TinyGPS code.
This was a "No Brainer" I just read all the documentation first... Something that seems to be missing here..    EDIT Doc

Doc


Thank you. Where did you find that documentation?
It looks like the doc I have not been able to find.

petterg


Quote
What is the disadvantages of increasing the buffer size?

In my opinion, that allows for sloppy coding. There are situations where it is unavoidable, I guess. The biggest drawback is that there limited memory available, and there is only one value that defines the size of the incoming AND outgoing serial buffers.

Still, it may be necessary, in your case, to make the change.


I agree the coding is not prefered. But if the alternative is to copy buffer content into a variable for "level2-buffering", increased buffer sounds less bad.
That input and output buffer can't be set individually is a downside though.

mikalhart

Just a couple of observations that might be useful here:

Quote
If the encode() takes longer to process than the time between sentences that will still cause problems.

encode() is very fast in both TinyGPS and TinyGPS++.  Using a static simulated data feed, we've measured that either library can process hundreds of sentences per second without issue.

What's more likely is that delays in the main body of the code are clogging things up, although it's not clear that's even happening here.  Fortunately, it's easy to detect this condition by examining whether the TinyGPS[++] object is experiencing an increasing checksum failures.  Also, if you're using SoftwareSerial, look to see if you're experiencing any overflow().  If checksums are not increasing, then your sketch is processing the incoming stream perfectly well.  There should be no need to buffer anything.

Tip: never use delay() in a TinyGPS[++] sketch.

Go Up