Arduino Forum

Using Arduino => Programming Questions => Topic started by: petterg on Oct 03, 2013, 02:22 am

Title: GPS: set update frequency
Post by: petterg on Oct 03, 2013, 02:22 am
Hi guys

I'm quite new to arduino (started with the blink example yesterday) so please have patience if this seems totally noob.

I've got a Skylab skm53 GPS hooked up to a Pro micro. Running the example code for TinyGPS (changed to hardware serial) I've got the GPS working. But it's only updating the long-/latitude every 3-5 minutes. I won't brag about it, but I'm able to move quite some distance between those updates!
The code is printing out data every second and I see the timestamp is updating every second, but my speed, direction and position remains the same for 3-5 minutes. When those data change they might do 3-5 updates within 3-5 seconds, then they reminds the same for another 3-5 minutes.

If I manage to read the datasheet right the gps should be able to do 10 updates per second. So how can I make this thing update position more frequently?

To test I went out on a field, far from any possible disturbance, to no avail. The gps never reported it had contact with less than 9 satellites.

Code:
https://github.com/awaxa/arduino/blob/master/libraries/TinyGPS/Examples/test_with_gps_device/test_with_gps_device.pde

Datasheet
http://www.webtronico.com/documentos/SKM53_DS_030609.pdf
Title: Re: GPS: set update frequency
Post by: int2str on Oct 03, 2013, 02:35 am
Did you let the GPS run for a while?
Do you have clear line of sight to the sky?

Sounds like maybe just bad reception?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 03, 2013, 11:35 am
It ran for about 6 hours.
The only place I can think of with a better sight than out in the field would be the top of a mountain.

During those 6 hours it was indoors half the time - my laptop needed to charge. The circuit is powered from the laptops usb. Indoors it had connection to 9 satellites. Outdoors it had between 9 and 12.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 03, 2013, 11:50 am
Quote
Running the example code for TinyGPS (changed to hardware serial)

So, you posted a link to some other code. Would you like a link to some other answer?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 03, 2013, 01:15 pm
I though it would be good practice for the forum to use a link rather than posting the code. A link to the solution would be fine though.
Here's the modified code: (replaced "nss" with "Serial1", commented out softwareserial)

Code: [Select]

//#include <SoftwareSerial.h>

#include <TinyGPS.h>

/* This sample code demonstrates the normal use of a TinyGPS object.
   It requires the use of SoftwareSerial, and assumes that you have a
   4800-baud serial GPS device hooked up on pins 3(rx) and 4(tx).
*/

TinyGPS gps;
//SoftwareSerial nss(3, 4);

static void gpsdump(TinyGPS &gps);
static bool feedgps();
static void print_float(float val, float invalid, int len, int prec);
static void print_int(unsigned long val, unsigned long invalid, int len);
static void print_date(TinyGPS &gps);
static void print_str(const char *str, int len);

void setup()
{
  Serial.begin(115200);
  Serial1.begin(9600);
 
  Serial.print("Testing TinyGPS library v. "); Serial.println(TinyGPS::library_version());
  Serial.println("by Mikal Hart");
  Serial.println();
  Serial.print("Sizeof(gpsobject) = "); Serial.println(sizeof(TinyGPS));
  Serial.println();
  Serial.println("Sats HDOP Latitude Longitude Fix  Date       Time       Date Alt     Course Speed Card  Distance Course Card  Chars Sentences Checksum");
  Serial.println("          (deg)    (deg)     Age                        Age  (m)     --- from GPS ----  ---- to London  ----  RX    RX        Fail");
  Serial.println("--------------------------------------------------------------------------------------------------------------------------------------");
}

void loop()
{
  bool newdata = false;
  unsigned long start = millis();
 
  // Every second we print an update
  while (millis() - start < 1000)
  {
    if (feedgps())
      newdata = true;
  }
 
  gpsdump(gps);
}

static void gpsdump(TinyGPS &gps)
{
  float flat, flon;
  unsigned long age, date, time, chars = 0;
  unsigned short sentences = 0, failed = 0;
  static const float LONDON_LAT = 51.508131, LONDON_LON = -0.128002;
 
  print_int(gps.satellites(), TinyGPS::GPS_INVALID_SATELLITES, 5);
  print_int(gps.hdop(), TinyGPS::GPS_INVALID_HDOP, 5);
  gps.f_get_position(&flat, &flon, &age);
  print_float(flat, TinyGPS::GPS_INVALID_F_ANGLE, 9, 5);
  print_float(flon, TinyGPS::GPS_INVALID_F_ANGLE, 10, 5);
  print_int(age, TinyGPS::GPS_INVALID_AGE, 5);

  print_date(gps);

  print_float(gps.f_altitude(), TinyGPS::GPS_INVALID_F_ALTITUDE, 8, 2);
  print_float(gps.f_course(), TinyGPS::GPS_INVALID_F_ANGLE, 7, 2);
  print_float(gps.f_speed_kmph(), TinyGPS::GPS_INVALID_F_SPEED, 6, 2);
  print_str(gps.f_course() == TinyGPS::GPS_INVALID_F_ANGLE ? "*** " : TinyGPS::cardinal(gps.f_course()), 6);
  print_int(flat == TinyGPS::GPS_INVALID_F_ANGLE ? 0UL : (unsigned long)TinyGPS::distance_between(flat, flon, LONDON_LAT, LONDON_LON) / 1000, 0xFFFFFFFF, 9);
  print_float(flat == TinyGPS::GPS_INVALID_F_ANGLE ? 0.0 : TinyGPS::course_to(flat, flon, LONDON_LAT, LONDON_LON), TinyGPS::GPS_INVALID_F_ANGLE, 7, 2);
  print_str(flat == TinyGPS::GPS_INVALID_F_ANGLE ? "*** " : TinyGPS::cardinal(TinyGPS::course_to(flat, flon, LONDON_LAT, LONDON_LON)), 6);

  gps.stats(&chars, &sentences, &failed);
  print_int(chars, 0xFFFFFFFF, 6);
  print_int(sentences, 0xFFFFFFFF, 10);
  print_int(failed, 0xFFFFFFFF, 9);
  Serial.println();
}

static void print_int(unsigned long val, unsigned long invalid, int len)
{
  char sz[32];
  if (val == invalid)
    strcpy(sz, "*******");
  else
    sprintf(sz, "%ld", val);
  sz[len] = 0;
  for (int i=strlen(sz); i<len; ++i)
    sz[i] = ' ';
  if (len > 0)
    sz[len-1] = ' ';
  Serial.print(sz);
  feedgps();
}

static void print_float(float val, float invalid, int len, int prec)
{
  char sz[32];
  if (val == invalid)
  {
    strcpy(sz, "*******");
    sz[len] = 0;
        if (len > 0)
          sz[len-1] = ' ';
    for (int i=7; i<len; ++i)
        sz[i] = ' ';
    Serial.print(sz);
  }
  else
  {
    Serial.print(val, prec);
    int vi = abs((int)val);
    int flen = prec + (val < 0.0 ? 2 : 1);
    flen += vi >= 1000 ? 4 : vi >= 100 ? 3 : vi >= 10 ? 2 : 1;
    for (int i=flen; i<len; ++i)
      Serial.print(" ");
  }
  feedgps();
}

static void print_date(TinyGPS &gps)
{
  int year;
  byte month, day, hour, minute, second, hundredths;
  unsigned long age;
  gps.crack_datetime(&year, &month, &day, &hour, &minute, &second, &hundredths, &age);
  if (age == TinyGPS::GPS_INVALID_AGE)
    Serial.print("*******    *******    ");
  else
  {
    char sz[32];
    sprintf(sz, "%02d/%02d/%02d %02d:%02d:%02d   ",
        month, day, year, hour, minute, second);
    Serial.print(sz);
  }
  print_int(age, TinyGPS::GPS_INVALID_AGE, 5);
  feedgps();
}

static void print_str(const char *str, int len)
{
  int slen = strlen(str);
  for (int i=0; i<len; ++i)
    Serial.print(i<slen ? str[i] : ' ');
  feedgps();
}

static bool feedgps()
{
  while (Serial1.available())
  {
    if (gps.encode(Serial1.read()))
      return true;
  }
  return false;
}
Title: Re: GPS: set update frequency
Post by: MarkT on Oct 03, 2013, 01:29 pm

I though it would be good practice for the forum to use a link rather than posting the code.


Good practice is explained in detail in the sticky threads...  Its there to facilitate rapid
and accurate solutions to peoples problems and reduce the to-ing and fro-ing of questions
and answers and wasted effort in answering a different problem from the actual problem :)
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 03, 2013, 01:48 pm
Code: [Select]
  // Every second we print an update
  while (millis() - start < 1000)
  {
    if (feedgps())
      newdata = true;
  }

That code does not do what the comment says.

Try this:
Code: [Select]
if(feedgps())
   gosdump(gps);

Nothing else in loop().
Title: Re: GPS: set update frequency
Post by: petterg on Oct 03, 2013, 05:23 pm
That helped quite a bit!
Now it's printing position about 3 times pr second, continuously updating for a while (30-60 secondes) - then it stop updating for a couple of minutes. Then it starts updating again for a another 30-60 seconds. ...And so on.

Could it be that the serial read comes out of sync with the data stream? Does the serial buffer need to be read continuously?
Could the solution be to flush buffer and wait for the start of a new sentence in order to start a new read? If so, how would the pro micro recognize the start of a new sentence?

(I don't fully understand every part of the sample code yet.)
Title: Re: GPS: set update frequency
Post by: MarkT on Oct 03, 2013, 05:49 pm
Have you thought of buffering an entire NMEA sentence before passing to the library?  Then
it won't matter if the libraries slow in parsing it.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 03, 2013, 06:26 pm
Quote
Could the solution be to flush buffer

No.

What is the purpose of sending data TO the GPS? What do you expect it to do with it?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 03, 2013, 06:28 pm

Have you thought of buffering an entire NMEA sentence before passing to the library?  Then
it won't matter if the libraries slow in parsing it.


That could be an idea. How would I start/stop the buffering?
How does the buffer work? Will it automatically throw away old data when full, or does it drop new data when full? Does it receive data in the background independently of what code the mpu is currently executing?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 03, 2013, 06:30 pm

Quote
Could the solution be to flush buffer

No.

What is the purpose of sending data TO the GPS? What do you expect it to do with it?


The purpose would be to say "Hey, I'd like some data now. Send me a sentence."
Then I would know that what is in the buffer is one full sentence ready for parsing/decoding.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 03, 2013, 06:43 pm
Quote
Then I would know that what is in the buffer is one full sentence ready for parsing/decoding.

You know that the GPS has sent a complete sentence because feedgps() returned true.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 03, 2013, 08:06 pm

Quote
Then I would know that what is in the buffer is one full sentence ready for parsing/decoding.

You know that the GPS has sent a complete sentence because feedgps() returned true.


Ah! That is useful knowledge!

I found this comment about the tinygps library:
Quote

NEMA format GPS data is usually transmitted at 4800 baud, in 8N1 format. The resulting maximum incoming speed is 480 bytes per second. By default, Teensyduino's UART Serial buffers up to 64 bytes, or 0.133 seconds of incoming data.

If your program must perform some operation for more than 0.13 seconds, it should check for new data and give it to TinyGPS with gps.encode() within that time period.


This gps run at 9600 baud. That would make the buffer fill every 0.0665 second.
As the position is printed to pc about 3 times pr second after PaulS' suggested modification to loop {}, the gps.encode() seems to need 0.3 seconds to run. That is 4.5 times longer than the buffer fill. I guess that  would be the cause of trouble.

So, if flushing buffer is not the way to handle this, then what is?

Quote
Could the solution be to flush buffer

No.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 03, 2013, 08:13 pm
Quote
As the position is printed to pc about 3 times pr second after PaulS' suggested modification to loop {}, the gps.encode() seems to need 0.3 seconds to run. That is 4.5 times longer than the buffer fill. I guess that  would be the cause of trouble.

How are you measuring that? The gps.encode() function get called only when a complete sentence has arrived. Since that happens roughly once a second, it should not cause the buffer to overflow, since the GPS will be quiet for a while after the end of the sentence has arrived, before it starts sending the next one.

Doubling the baud rate halves the time it takes to receive the data, not doubles it.
Title: Re: GPS: set update frequency
Post by: Brad Burleson on Oct 03, 2013, 09:25 pm

Here's the modified code: (replaced "nss" with "Serial1", commented out softwareserial)


You'd be much better off looking at the example code provided with TinyGPS as opposed to someone else's modified version.

Check out the examples included at http://arduiniana.org/libraries/tinygps/ (http://arduiniana.org/libraries/tinygps/).

FWIW,

Brad
KF7FER

Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 12:42 am

How are you measuring that?

date/time is part of the printed information. I'm counting 3 lines with the same seconds value, hence I know the printing is performed about 3 times per second.


The gps.encode() function get called only when a complete sentence has arrived. Since that happens roughly once a second, it should not cause the buffer to overflow, since the GPS will be quiet for a while after the end of the sentence has arrived, before it starts sending the next one.

How do you know it's sending data only once per second?
As far as I understand the datasheet it's sending 10 times per second.


Doubling the baud rate halves the time it takes to receive the data, not doubles it.


I think 0.0665 is closer to half than double of 0.13.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 12:44 am

You'd be much better off looking at the example code provided with TinyGPS as opposed to someone else's modified version.

Check out the examples included at http://arduiniana.org/libraries/tinygps/ (http://arduiniana.org/libraries/tinygps/).


Thanks. I didn't know there was an official site for the library.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 04, 2013, 01:25 am
Quote
As far as I understand the datasheet it's sending 10 times per second.

Where did you get that? The link in your first post says, on the middle of page 2:
Quote
Navigation Data Update Rate 1Hz

That's once a second.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 02:32 am

Quote
As far as I understand the datasheet it's sending 10 times per second.

Where did you get that? The link in your first post says, on the middle of page 2:
Quote
Navigation Data Update Rate 1Hz

That's once a second.


I read 10Hz somewhere. Might have been some other place.



Check out the examples included at http://arduiniana.org/libraries/tinygps/ (http://arduiniana.org/libraries/tinygps/).


Thanks to that link I found the library TinyGPS+. Thing speeded up quite a bit using that. Printing is now 7-10 times per second. Position updates are still lagging a bit thou. If I walk, stop, walk, stop, the printout shows exactly that I've walked, stopped, walked, stopped. It has a delay of about 15-20 seconds. I didn't get to test this very well yet as the weather is not equipment friendly at the moment (windy and rain). How can I figure out whats causing the delay? Is it the gps, the buffer or something else?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 01:29 pm
From http://arduino.cc/en/Serial/Flush I read:
Quote

Waits for the transmission of outgoing serial data to complete. (Prior to Arduino 1.0, this instead removed any buffered incoming serial data.)


What's the best way now to empty the buffer for incoming data like flush() did prior to Arduino 1.0?

(For testing I'd like to empty the buffer prior to reading data to make sure it's dealing with fresh data)
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 01:43 pm


Quote
As far as I understand the datasheet it's sending 10 times per second.

Where did you get that? The link in your first post says, on the middle of page 2:
Quote
Navigation Data Update Rate 1Hz

That's once a second.


I read 10Hz somewhere. Might have been some other place.


Guess I mixed up the bandwidth and navigation data update rate
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 04, 2013, 02:56 pm
Quote
Printing is now 7-10 times per second.

Why? Once a second the GPS tells you where you are. Why would you print the same data 7 to 10 times?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 05:52 pm

Quote
Printing is now 7-10 times per second.

Why? Once a second the GPS tells you where you are. Why would you print the same data 7 to 10 times?


I'm telling what it does. If it did what I wanted there wouldn't be any need for this thread. I guess it's parsing/printing as long as there is data to parse/print. I'd like it to process THE LATEST data only when needed. I consider it pointless to spend cpu time to process data continuously just to trow away the results. I also consider it pointless to let the buffer fill up, and process old data when a reading is wanted. Hence my questions how the buffer behaves when full, how to throw away all buffered data without processing (to make sure the next data to read and process is fresh), or how to make the gps only send data when it will be used for something., or as topic says: how to control how often the gps send updates.
Title: Re: GPS: set update frequency
Post by: wildbill on Oct 04, 2013, 06:22 pm
Time to post your code by the sound of it.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 07:26 pm
From http://arduiniana.org/libraries/tinygpsplus/
Example code, edited to use Serial1 at 9600 baud

Code: [Select]

#include <TinyGPS++.h>
//#include <SoftwareSerial.h>
/*
   This sample sketch demonstrates the normal use of a TinyGPS++ (TinyGPSPlus) object.
   It requires the use of SoftwareSerial, and assumes that you have a
   4800-baud serial GPS device hooked up on pins 4(rx) and 3(tx).
*/
//static const int RXPin = 4, TXPin = 3;
static const int GPSBaud = 9600;

// The TinyGPS++ object
TinyGPSPlus gps;

// The serial connection to the GPS device
//SoftwareSerial ss(RXPin, TXPin);

void setup()
{
  Serial.begin(115200);
  Serial1.begin(GPSBaud);

  Serial.println(F("DeviceExample.ino"));
  Serial.println(F("A simple demonstration of TinyGPS++ with an attached GPS module"));
  Serial.print(F("Testing TinyGPS++ library v. ")); Serial.println(TinyGPSPlus::libraryVersion());
  Serial.println(F("by Mikal Hart"));
  Serial.println();
}

void loop()
{
  // This sketch displays information every time a new sentence is correctly encoded.
  while (Serial1.available() > 0)
    if (gps.encode(Serial1.read()))
      displayInfo();

  if (millis() > 5000 && gps.charsProcessed() < 10)
  {
    Serial.println(F("No GPS detected: check wiring."));
    while(true);
  }
}

void displayInfo()
{
  Serial.print(F("Location: "));
  if (gps.location.isValid())
  {
    Serial.print(gps.location.lat(), 6);
    Serial.print(F(","));
    Serial.print(gps.location.lng(), 6);
  }
  else
  {
    Serial.print(F("INVALID"));
  }

  Serial.print(F("  Date/Time: "));
  if (gps.date.isValid())
  {
    Serial.print(gps.date.month());
    Serial.print(F("/"));
    Serial.print(gps.date.day());
    Serial.print(F("/"));
    Serial.print(gps.date.year());
  }
  else
  {
    Serial.print(F("INVALID"));
  }

  Serial.print(F(" "));
  if (gps.time.isValid())
  {
    if (gps.time.hour() < 10) Serial.print(F("0"));
    Serial.print(gps.time.hour());
    Serial.print(F(":"));
    if (gps.time.minute() < 10) Serial.print(F("0"));
    Serial.print(gps.time.minute());
    Serial.print(F(":"));
    if (gps.time.second() < 10) Serial.print(F("0"));
    Serial.print(gps.time.second());
    Serial.print(F("."));
    if (gps.time.centisecond() < 10) Serial.print(F("0"));
    Serial.print(gps.time.centisecond());
  }
  else
  {
    Serial.print(F("INVALID"));
  }

  Serial.println();
}
Title: Re: GPS: set update frequency
Post by: wildbill on Oct 04, 2013, 07:51 pm
This looks interesting:
Code: [Select]
  // This sketch displays information every time a new sentence is correctly encoded.

I wonder how many different sentences your device is emitting. I'd suggest you copy that sketch and reduce it to a simpler one that just echoes whatever is read from the gps to serial so you can find out.
Title: Re: GPS: set update frequency
Post by: Brad Burleson on Oct 04, 2013, 08:27 pm

I wonder him many different sentences your device is emitting. I'd suggest you copy that sketch and reduce it to a simpler one that just echoes whatever is read from the gps to serial so you can find out.


Also if you read the "Usage" section on the TinyGPS++ page you'll see there are new methods that the example code isn't using, such as:

Code: [Select]

if (gps.altitude.isUpdated())
   Serial.println(gps.altitude.meters());


Might be an easier way to go.  Or if you're so inclined you should be able to tell the GPS what sentences to output and that would help as well (or at least decrease the number of updates you see).

Regards,

Brad
KF7FER
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 04, 2013, 10:43 pm
At the very least. sharing the serial output would be useful. According to the datasheet, each sentence is updated once a second. There may, as has been mentioned, more than once sentence type. We can't see what you are seeing.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 10:44 pm

This looks interesting:
Code: [Select]
 // This sketch displays information every time a new sentence is correctly encoded.

I wonder how many different sentences your device is emitting. I'd suggest you copy that sketch and reduce it to a simpler one that just echoes whatever is read from the gps to serial so you can find out.


That was a good idea!
Edit2: bug fixed. Fixed code/results to follow

Running code:
Code: [Select]

void loop()
{
 while (Serial1.available() > 0 && i < 5000) {
   i++;
   Serial.print(Serial1.read());
 }
 if(i < 5000)
   Serial.println("next");
 delay(50);
}



Edit 3: human readable output posted in this post.


So it seems like the gps is sending 7 sentences, and there is a delay of 12x(50ms+looptime) between the blocks of sentences. With 7x 50ms during the sentences this backs up the datasheets/PaulS statement of update frequency of 1Hz.

How many of these sentences is the buffer able to store?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 11:04 pm

Also if you read the "Usage" section on the TinyGPS++ page you'll see there are new methods that the example code isn't using, such as:

Code: [Select]

if (gps.altitude.isUpdated())
   Serial.println(gps.altitude.meters());


Using the isUpdated() does require that encode() is ran before. If the encode() takes longer to process than the time between sentences that will still cause problems.


Might be an easier way to go.  Or if you're so inclined you should be able to tell the GPS what sentences to output and that would help as well (or at least decrease the number of updates you see).


With the observation with the echo-sketch I totally agree that reducing the number of sentences will make a significant performance boots. But how?
Title: Re: GPS: set update frequency
Post by: wildbill on Oct 04, 2013, 11:20 pm
You'll need to decide which sentences to pass to the tinyGPS encoder and throw the rest away.

Figuring out which you want would be easier if you cast the int you're getting from read to char thus:
Code: [Select]

    Serial.print((char)Serial1.read());


You could likely get rid of all the code relating to 5000 in this circumstance too.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 04, 2013, 11:39 pm

You'll need to decide which sentences to pass to the tinyGPS encoder and throw the rest away.

Figuring out which you want would be easier if you cast the int you're getting from read to char thus:
Code: [Select]

    Serial.print((char)Serial1.read());


You could likely get rid of all the code relating to 5000 in this circumstance too.


The reason for the 5000 is to make it possible to copy the text from the serial monitor. I'm to slow to mark and copy within 50ms.


Casted result:
Code: [Select]
next
next
next
next
next
$GPGGA,213127.000,5915.5411,N,01027.5115,E,1,8next
,1.23,0.6,M,41.0,M,,*53
$GPGSA,A,3,20,04,10,23,0next
2,07,09,08,,,,,1.52,1.23,0.90*0F
$GPGSV,4,1,13,1next
3,73,093,,10,72,249,16,23,42,083,26,02,42,270,26next
*73
$GPGSV,4,2,13,07,42,174,19,04,32,216,22,16,2next
4,063,,05,22,296,*7A
$GPGSV,4,3,13,33,19,209,,0next
8,14,189,24,29,14,336,,09,12,197,21*7A
$GPGSV,4,next
4,13,20,05,131,17*49
$GPRMC,213127.000,A,5915.54next
18,N,01027.5115,E,0.00,261.16,041013,,,A*6F
next
next
next
next
next
next
next
next
next
next
next
$GPGGA,213128.000,5915.5411,next
N,01027.5115,E,1,8,1.23,0.6,M,41.0,M,,*5C
$GPGSAnext
,A,3,20,04,10,23,02,07,09,08,,,,,1.52,1.23,0.90*next
0F
$GPGSV,4,1,13,13,73,093,,10,72,249,16,23,42,0next
83,26,02,42,270,26*73
$GPGSV,4,2,13,07,42,174,19next
,04,32,216,22,16,24,063,,05,22,296,*7A
$GPGSV,4next
,3,13,33,19,209,,08,14,189,24,29,14,336,,09,12,19next
7,21*7A
$GPGSV,4,4,13,20,05,131,17*49
$GPRMC,2next
13128.000,A,5915.5411,N,01027.5115,E,0.00,261.16,next
041013,,,A*60
next
next
next
next
next
next
next
next
next
next
next
$GPGGA,213129.000,5next
915.5411,N,01027.5115,E,1,8,1.23,0.6,M,41.0,M,,*next
5D
$GPGSA,A,3,20,04,10,23,02,07,09,08,,,,,1.52,1next
.23,0.90*0F
$GPGSV,4,1,13,13,73,093,,10,72,249,1next
6,23,42,083,26,02,42,270,26*73
$GPGSV,4,2,13,07next
,42,174,19,04,32,216,23,16,24,063,,05,23,296,*7A
next

$GPGSV,4,3,13,33,19,209,,08,14,189,24,29,14,336,next
,09,12,197,21*7A
$GPGSV,4,4,13,20,05,131,17*49
next

$GPRMC,213129.000,A,5915.5411,N,01027.5115,E,0.0next
0,261.16,041013,,,A*61
next
next
next
next
next
next
next
next
next
next
next
$GPGGA,213130.000,5915.5411,N,0next
1027.5115,E,1,8,1.23,0.6,M,41.0,M,,*55
$GPGSA,Anext
,3,20,04,10,23,02,07,09,08,,,,,1.52,1.23,0.90*0F
next

$GPGSV,4,1,13,13,73,093,,10,72,249,16,23,42,083next
,27,02,42,270,26*72
$GPGSV,4,2,13,07,42,174,19,0next
4,32,216,23,16,24,063,,05,23,296,*7A
$GPGSV,4,3,next
13,08,14,189,24,29,14,336,,09,12,197,21,40,09,12next
2,*75
$GPGSV,4,4,13,20,05,131,18*46
$GPRMC,2131next
30.000,A,5915.5411,N,01027.5115,E,0.00,261.16,041next
013,,,A*69
next
next
next
next
next
next
next
next
next
next
next
$GPGGA,213131.000,5915.5411,N,01027.5115,Enext
,1,8,1.23,0.6,M,41.0,M,,*54
$GPGSA,A,3,20,04,10next
,23,02,07,09,08,,,,,1.52,1.23,0.90*0F
$GPGSV,4,1next
,13,13,73,093,,10,72,249,16,23,42,083,27,02,42,27next
0,25*71
$GPGSV,4,2,13,07,42,174,18,04,32,216,23next
,16,24,063,,05,23,296,*7B
$GPGSV,4,3,13,08,14,18next
9,24,29,14,336,,09,12,197,21,40,09,122,*75
$GPGnext
SV,4,4,13,20,05,131,18*46
$GPRMC,213131.000,A,59next
15.5411,N,01027.5115,E,0.00,261.16,041013,,,A*68
next

next
next
next
next
next
next
next
next
next
next
$GPGGnext
A,213132.000,5915.5411,N,01027.5115,E,1,8,1.23,0.next
6,M,41.0,M,,*57
$GPGSA,A,3,20,04,10,23,02,07,09next
,08,,,,,1.52,1.23,0.90*0F
$GPGSV,4,1,13,13,73,09next
3,,10,72,249,16,23,42,083,27,02,42,270,25*71
$GPnext
GSV,4,2,13,07,42,174,18,04,32,216,23,16,24,063,,next
05,23,296,*7B
$GPGSV,4,3,13,08,14,189,24,29,14,3next
36,,09,12,197,21,40,09,122,*75
$GPGSV,4,4,13,20next
,05,131,18*46
$GPRMC,213132.000,A,5915.5411,N,01next
027.5115,E,0.00,261.16,041013,,,A*6B
next
next
next
next
next
next
next
next
next
next
next
$GPGGA,213133.000,5next
915.5411,N,01027.5115,E,1,8,1.23,0.6,M,41.0,M,,*5next
6
$GPGSA,A,3,20,04,10,23,02,07,09,08,,,,,1.52,1next
.23,0.90*0F
$GPGSV,4,1,13,13,73,093,,10,72,249,1next
6,23,42,083,27,02,42,270,25*71
$GPGSV,4,2,13,07,next
42,174,18,04,32,216,23,16,24,063,,05,23,296,*7B
next

$GPGSV,4,3,13,08,14,189,23,29,14,336,,09,12,197,next
21,40,09,122,*72
$GPGSV,4,4,13,20,05,131,18*46
next

$GPRMC,213133.000,A,5915.5411,N,01027.5115,E,0.0next
0,261.16,041013,,,A*6A
next
next
next
next
next
next
next
next
next
next
$next
GPGGA,213134.000,5915.5411,N,01027.5115,E,1,8,1.next
23,0.6,M,41.0,M,,*51
$GPGSA,A,3,20,04,10,23,02,0next
7,09,08,,,,,1.52,1.23,0.90*0F
$GPGSV,4,1,13,13,7next
3,093,,10,72,248,17,23,42,083,27,02,42,270,25*71next

$GPGSV,4,2,13,07,42,174,18,04,32,216,24,16,24,0next
63,,


Any way to make it send only one of the senteces?
Title: Re: GPS: set update frequency
Post by: wildbill on Oct 04, 2013, 11:59 pm
I found a manual for the device which says this:
Quote
The SkyNav SKM53 module supports the following NMEA-0183 messages: GGA, GLL, GSA, GSV, RMC VTG, ZDA and DTM. The module default NMEA-0183 output is set up GGA, GSA ,GSV, RMC and default baud rate is set up 9600bps


Which kind of implies that it can be set to emit a different set of sentences. No clue how in that document though - do you have any better paperwork? The alternative of course is to do some initial parsing of the data yourself and only pass the sentences you're interested in to the encoder.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 05, 2013, 12:06 am

I found a manual for the device which says this:
Quote
The SkyNav SKM53 module supports the following NMEA-0183 messages: GGA, GLL, GSA, GSV, RMC VTG, ZDA and DTM. The module default NMEA-0183 output is set up GGA, GSA ,GSV, RMC and default baud rate is set up 9600bps


Which kind of implies that it can be set to emit a different set of sentences. No clue how in that document though - do you have any better paperwork? The alternative of course is to do some initial parsing of the data yourself and only pass the sentences you're interested in to the encoder.


That document is the only I have. Maybe there is some standard defining how to send instructions to any gps chip?
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 05, 2013, 12:23 am
Quote
Maybe there is some standard defining how to send instructions to any gps chip?

There is not. Contact the manufacturer. Ask them how to tell the GPS to send only useful information.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 05, 2013, 01:24 am
Mail sent to Skylab, requesting more documentation.
Title: Re: GPS: set update frequency
Post by: Brad Burleson on Oct 05, 2013, 02:44 am

Using the isUpdated() does require that encode() is ran before.


Of course.  I was just showing an easy way to know when you had updated data (but not necessarily changed).

Quote

If the encode() takes longer to process than the time between sentences that will still cause problems.


I really doubt that is your problem.  I've never used the new version but I've done quite a bit of work with the older versions and never had that sort of issue (and I was one of the original beta testers).

BTW I'd ditch the delay and the "next"... can you see how "next" is being printed before the end of each line?  (each sentence should end with an "*" and a two digit checksum followed by the end of line character(s)).  

As an example, one complete sentence is being output like this:

Code: [Select]
$GPGGA,213127.000,5915.5411,N,01027.5115,E,1,8next
,1.23,0.6,M,41.0,M,,*53


Regards,

Brad
KF7FER

EDIT: I should have added that if you want to capture the output of your GPS, don't use the serial monitor - use a terminal emulator and simply open it's capture buffer before you power up your arduino.  Plus you don't have to cut and paste :-)
Title: Re: GPS: set update frequency
Post by: petterg on Oct 05, 2013, 01:15 pm


Using the isUpdated() does require that encode() is ran before.


Of course.  I was just showing an easy way to know when you had updated data (but not necessarily changed).

Quote

If the encode() takes longer to process than the time between sentences that will still cause problems.


I really doubt that is your problem.


What would you think the cause of problem could be?
All I really need in "real time" is a "way point reached" warning. Basically read the position, compare to the position of a way point, make a beep when current position is within a given distance from the way point. The best I've got so far is a 20 second delay, which in best case would give a "way point passed 20 seconds ago" warning.



I've never used the new version but I've done quite a bit of work with the older versions and never had that sort of issue (and I was one of the original beta testers).

Maybe you happen to know how to change which messages that the gps module outputs then?


BTW I'd ditch the delay and the "next"... can you see how "next" is being printed before the end of each line?  (each sentence should end with an "*" and a two digit checksum followed by the end of line character(s)). 

As an example, one complete sentence is being output like this:

Code: [Select]
$GPGGA,213127.000,5915.5411,N,01027.5115,E,1,8next
,1.23,0.6,M,41.0,M,,*53


Thanks for pointing that out.
As far as I can tell, the fact that "next" occur in the middle of an sentence also indicates that "Serial1.available() > 0" may return false while a sentence is being transmitted. To me that is an unexpected behavior. Does this indicate that the buffer is full?


use a terminal emulator

Thanks. I have a lot to learn about this.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 05, 2013, 01:30 pm
Quote
Basically read the position, compare to the position of a way point, make a beep when current position is within a given distance from the way point. The best I've got so far is a 20 second delay, which in best case would give a "way point passed 20 seconds ago" warning.

We've posted corrections to your code. You've said that you've integrated them. But, clearly, you've gone a lot further than that. Now, you are having problems with the code you haven't posted. Can you, perhaps thinking outside the box, come up with a solution to our dilemma?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 05, 2013, 01:39 pm

Quote
Basically read the position, compare to the position of a way point, make a beep when current position is within a given distance from the way point. The best I've got so far is a 20 second delay, which in best case would give a "way point passed 20 seconds ago" warning.

We've posted corrections to your code. You've said that you've integrated them. But, clearly, you've gone a lot further than that. Now, you are having problems with the code you haven't posted. Can you, perhaps thinking outside the box, come up with a solution to our dilemma?


I'm sorry for telling you why I want to read position with less than 20 seconds delay. It must have been very confusing.


Code was posted here
Title: Re: GPS: set update frequency
Post by: wildbill on Oct 05, 2013, 01:40 pm
Quote
As far as I can tell, the fact that "next" occur in the middle of an sentence also indicates that "Serial1.available() > 0" may return false while a sentence is being transmitted. To me that is an unexpected behavior. Does this indicate that the buffer is full?


Serial data transmission is slow and asynchronous and 9600 is a pretty slow baud rate. The arduino is very quick by comparison; it can pull a character from the buffer and do a whole bunch of work before the next one arrives. It is a very very common misconception that you'll get all the serial data you want in one continuous read. If you're in the middle of a sentence and there's nothing to read, it does not mean that the buffer is full, on the contrary, it means it's empty.

If you're sending serial data, that's slow too and has a small buffer. If you send more than enough to fill the buffer, the arduino will wait until there's space in the buffer again. In this circumstance, the act of sending more than 64 (varies by arduino) characters can delay you enough that your serial reads appear to pull a whole packet at once. In the case of a GPS sentence though as they can be quite long, you may well fill the read buffer and start dropping incoming data during the send delay.

As to ignoring sentences, you may get lucky and get better data from the manufacturer. If not, another possibility is to take a look at the tinyGPS library. Without looking, I assume it's hardcoded to some extent as to which sentences it can parse. It might be plausible to create a hacked copy that only 'knows' the ones you want.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 05, 2013, 01:45 pm
Quote
I'm sorry for telling you why I want to read position with less than 20 seconds delay. It must have been very confusing.

You can stuff the attitude. The GPS is, you've said, giving you good data, as often as can reasonably be expected. You are then doing something with that data, and that something results in a long delay. If you want help solving the problem, we need to see what you are seeing. That is the code and the serial output.

Quote
Code was posted here

Here? Where the hell is "here"?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 05, 2013, 02:51 pm

Quote
As far as I can tell, the fact that "next" occur in the middle of an sentence also indicates that "Serial1.available() > 0" may return false while a sentence is being transmitted. To me that is an unexpected behavior. Does this indicate that the buffer is full?


Serial data transmission is slow and asynchronous and 9600 is a pretty slow baud rate. The arduino is very quick by comparison; it can pull a character from the buffer and do a whole bunch of work before the next one arrives. It is a very very common misconception that you'll get all the serial data you want in one continuous read. If you're in the middle of a sentence and there's nothing to read, it does not mean that the buffer is full, on the contrary, it means it's empty.


So if I get you right, the occurrences of "next" in the middle of a sentence shows the characters that the gps managed to put into the buffer during the 50ms delay + loop processing time?


If you're sending serial data, that's slow too and has a small buffer. If you send more than enough to fill the buffer, the arduino will wait until there's space in the buffer again.

The sender will wait if the receivers buffer is full?
Meaning, if the MPU is not reading the buffer fast enough, the gps will queue messages waiting for the MPU to free up buffer? Assuemed the GPS also has a buffer, this would totally explain why I see delayed data!


In this circumstance, the act of sending more than 64 (varies by arduino) characters can delay you enough that your serial reads appear to pull a whole packet at once. In the case of a GPS sentence though as they can be quite long, you may well fill the read buffer and start dropping incoming data during the send delay.

From this I think I learned that buffer behavior is to drop new data when full rather than saving the last 64 bytes received. Is there a way to flip this behavior so that when buffer is full, the oldest data is dropped, allowing new data to always be written?


As to ignoring sentences, you may get lucky and get better data from the manufacturer. If not, another possibility is to take a look at the tinyGPS library. Without looking, I assume it's hardcoded to some extent as to which sentences it can parse. It might be plausible to create a hacked copy that only 'knows' the ones you want.

Thats one possibility. Another could be to monitor the datastream, waiting for the start of the wanted message, then collect the full message and pass it to the encode().
Title: Re: GPS: set update frequency
Post by: petterg on Oct 05, 2013, 03:03 pm

You can stuff the attitude.

Mirrors can be harsh. If you can't stand what you see in the mirror, the target for improvement should not be the mirror.


The GPS is, you've said, giving you good data, as often as can reasonably be expected. You are then doing something with that data, and that something results in a long delay. If you want help solving the problem, we need to see what you are seeing. That is the code and the serial output.

The output looks exactly as expected, except for the delay of 20 seconds. It doesn't tell anything if you don't watch it in real time while moving around.


Quote
Code was posted here

Here? Where the hell is "here"?


In the quote, if you quote correctly, there is a statement with timestamp and link that will take you to the posting where the quote is from.
Title: Re: GPS: set update frequency
Post by: wildbill on Oct 05, 2013, 04:26 pm
Quote
So if I get you right, the occurrences of "next" in the middle of a sentence shows the characters that the gps managed to put into the buffer during the 50ms delay + loop processing time?

Yes.
Quote
The sender will wait if the receivers buffer is full?
Meaning, if the MPU is not reading the buffer fast enough, the gps will queue messages waiting for the MPU to free up buffer? Assuemed the GPS also has a buffer, this would totally explain why I see delayed data!

No. If the arduino is transmitting more data that it can buffer, it blocks until what's left will fit in the buffer. During that blocking, you may lose data from the GPS.
Quote
From this I think I learned that buffer behavior is to drop new data when full rather than saving the last 64 bytes received. Is there a way to flip this behavior so that when buffer is full, the oldest data is dropped, allowing new data to always be written?

Your understanding is correct. Changing it would require hacking the core libraries and even if you could, you would still have dropped data. Better to arrange it so that you process the serial data in a timely fashion.
Quote
Thats one possibility. Another could be to monitor the datastream, waiting for the start of the wanted message, then collect the full message and pass it to the encode().

Indeed.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 05, 2013, 06:52 pm

No. If the arduino is transmitting more data that it can buffer, it blocks until what's left will fit in the buffer. During that blocking, you may lose data from the GPS.


Ah. So you were aiming towards the serial.print() to the computer could lock up the processing so that Serial1.read() from the gps would meanwhile run late?

I've been looking into the TinyGPS++ library now. To me it looks like only GGA and RMC messages are recognized. It also looks like encode() performs just the same amount of processing even when the message is not among those recognized. There is room for improvements.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 16, 2013, 03:24 pm
I did not get any better doc from skylab, so I did some modifications to the library, making it not parse any messages if the first 5 letters indicate it's not among those two that the library supports. For a small set of sample data this improved processing time from 75ms to 48ms. (Testdata was feeded to the library directly from a variable, not from the gps.)

However, this did not solve the problem with delayed updates.

What I think happens is this:
Data is written from GPS to arduino serial buffer. Buffer is read, first message is parsed. While the first message is parsed the buffer goes full. Buffer is read, data is dropped, start of the second supported message is detected, but the second half of the second message is dropped because of buffer overflow. Hence end of message is not detected, so it keeps reading and parsing unsupported messages until an end of message is detected. Then it gets an checksum error and the message is dropped.
Over time this error builds up. The message containing position data is quite a bit longer than the one containing date/time data. Hence the message containing position data frequently suffers from buffer overflow and is dropped.

So whats the options to get around this? Some of the options I see is:
1) Increase the buffer size so that it will be able to contain one sample of each message sent from the gps. That would give the library 1 second figure what to drop or parse.
2) Change the buffer behavior so that it's dropping the oldest data when full rather than not receiving new data.
3) Write the code so that it's somehow synchronized with the gps. Then it would be able to empty the buffer just before the interesting messages arrives.
4) Get an arduino due, which would parse the data faster, hopefully fast enough to keep the buffer from overflowing

What do you think of these ideas?
Does anyone have other ideas?
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 16, 2013, 03:46 pm
Quote
1) Increase the buffer size so that it will be able to contain one sample of each message sent from the gps. That would give the library 1 second figure what to drop or parse.

That might work, but it shouldn't take a whole second to parse the data that you got.

Quote
2) Change the buffer behavior so that it's dropping the oldest data when full rather than not receiving new data.

That's going to be a lot harder than you think.

Quote
3) Write the code so that it's somehow synchronized with the gps. Then it would be able to empty the buffer just before the interesting messages arrives.

Hmmm...

Quote
For a small set of sample data this improved processing time from 75ms to 48ms.

This suggests to me that you are not loosing data. New data shouldn't begin to arrive until the Arduino has had plenty of time to parse the last sentence collected.

Quote
4) Get an arduino due, which would parse the data faster, hopefully fast enough to keep the buffer from overflowing

Maybe. I haven't heard other people complaining about how long it takes to parse the data. And, the numbers you posted show it isn't taking you that long, either.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 16, 2013, 05:28 pm

Quote
1) Increase the buffer size so that it will be able to contain one sample of each message sent from the gps. That would give the library 1 second figure what to drop or parse.

That might work, but it shouldn't take a whole second to parse the data that you got.

The gps seems to be sending all sentences continuously until sent, then not send anything for the rest of the 1 second data transfer interval. So a buffer that will hold 1 second of data will need to have the same size as on that holds 0.3 seconds of data.


Quote
2) Change the buffer behavior so that it's dropping the oldest data when full rather than not receiving new data.

That's going to be a lot harder than you think.

Probably, yes....


Quote
3) Write the code so that it's somehow synchronized with the gps. Then it would be able to empty the buffer just before the interesting messages arrives.

Hmmm...

And I think, even if the sync-part should turn out doable, I doubt it would solve the problem.


Quote
For a small set of sample data this improved processing time from 75ms to 48ms.

This suggests to me that you are not loosing data. New data shouldn't begin to arrive until the Arduino has had plenty of time to parse the last sentence collected.

Well, yes and no. The full set of sentences arrive in one bulk. The issue is that parsing is performed as the sentences arrive. Assume those 48ms is 70% parsing position, 25% parsing date/time and 5% deciding the arriving message is not supported. That would imply that reading the buffer halts for 34ms during one block of sentences while parsing position.  If buffer is 64byte there will be an overflow if the data rate is more than 1882 bytes/second. What is the data rate for 9600baud serial? 9600byte/sec? If so, the buffer surely will overflow.


Quote
4) Get an arduino due, which would parse the data faster, hopefully fast enough to keep the buffer from overflowing

Maybe. I haven't heard other people complaining about how long it takes to parse the data. And, the numbers you posted show it isn't taking you that long, either.

If the assumption above is correct, processing at 16MHz*9600/1882=81MHz would solve the problem - IF the code is somewhat in sync with the gps, so that the code will not be busy doing other stuff when a new set of sentences start. After all, the target for the code is primarily to make use of the gps data, not just do the readings.
Title: Re: GPS: set update frequency
Post by: doughboy on Oct 16, 2013, 09:46 pm
probably not the answer you are looking for, but just throwing this out as an option.

can you try another gps? neo6m now costs like $17.
can you try other firmware besides tinygps? I know tinygps can be used in quadcopter equipped with gps, but the i2c_gps_nav program that provides similar functionality is more stable and reliable imho.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 17, 2013, 12:44 am
Quote
The gps seems to be sending all sentences continuously until sent, then not send anything for the rest of the 1 second data transfer interval.

Then, perhaps you need to clone the TinyGPS library so that you can copy the buffer that TinyGPS writes to when the end of the sentence that you want to parse arrives, and then, after the last sentence in the burst arrives, parse the copy. You should be done with the parsing by the time the next batch starts to arrive.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 17, 2013, 01:11 am

Quote
The gps seems to be sending all sentences continuously until sent, then not send anything for the rest of the 1 second data transfer interval.

Then, perhaps you need to clone the TinyGPS library so that you can copy the buffer that TinyGPS writes to when the end of the sentence that you want to parse arrives, and then, after the last sentence in the burst arrives, parse the copy. You should be done with the parsing by the time the next batch starts to arrive.


Thats very true.
Wouldn't that, in matter of resource usage, essentially be the same as increasing the buffer size? What is the disadvantages of increasing the buffer size?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 17, 2013, 01:14 am

can you try another gps? neo6m now costs like $17.

Why would that gps be better?


i2c_gps_nav

Thanks. I'll have a look at it.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 17, 2013, 01:20 am
Quote
What is the disadvantages of increasing the buffer size?

In my opinion, that allows for sloppy coding. There are situations where it is unavoidable, I guess. The biggest drawback is that there limited memory available, and there is only one value that defines the size of the incoming AND outgoing serial buffers.

Still, it may be necessary, in your case, to make the change.
Title: Re: GPS: set update frequency
Post by: Docedison on Oct 17, 2013, 03:49 am
Some time back... (the first of the year..) I was using a Skylabs SKM53... I had no problems connecting in the house. I had an update once a second.
I thought this might be helpful..
Code: [Select]
 
 // uncomment this line to turn on RMC (recommended minimum) and GGA (fix data) including altitude
 GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCGGA);
 // uncomment this line to turn on only the "minimum recommended" data
 //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);
 // For parsing data, we don't suggest using anything but either RMC only or RMC+GGA since
 // the parser doesn't care about other sentences at this time
 
 // Set the update rate
 GPS.sendCommand(PMTK_SET_NMEA_UPDATE_1HZ);   // 1 Hz update rate
 // For the parsing code to work nicely and have time to sort thru the data, and
 // print it out we don't suggest using anything higher than 1 Hz

 // the nice thing about this code is you can have a timer0 interrupt go off
 // every 1 millisecond, and read data from the GPS for you. that makes the
 // loop code a heck of a lot easier!

This from the Adafruit GPS tutorial and sketch... Using Mikal Hart's TinyGPS code.
This was a "No Brainer" I just read all the documentation first... Something that seems to be missing here..    EDIT Doc

Doc
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 17, 2013, 11:53 am
Quote
I just read all the documentation first

All of what documentation? Where did you get the snippet you posted?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 17, 2013, 08:55 pm

Some time back... (the first of the year..) I was using a Skylabs SKM53... I had no problems connecting in the house. I had an update once a second.
I thought this might be helpful..
Code: [Select]
 
 // uncomment this line to turn on RMC (recommended minimum) and GGA (fix data) including altitude
 GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCGGA);
 // uncomment this line to turn on only the "minimum recommended" data
 //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);
 // For parsing data, we don't suggest using anything but either RMC only or RMC+GGA since
 // the parser doesn't care about other sentences at this time
 
 // Set the update rate
 GPS.sendCommand(PMTK_SET_NMEA_UPDATE_1HZ);   // 1 Hz update rate
 // For the parsing code to work nicely and have time to sort thru the data, and
 // print it out we don't suggest using anything higher than 1 Hz

 // the nice thing about this code is you can have a timer0 interrupt go off
 // every 1 millisecond, and read data from the GPS for you. that makes the
 // loop code a heck of a lot easier!

This from the Adafruit GPS tutorial and sketch... Using Mikal Hart's TinyGPS code.
This was a "No Brainer" I just read all the documentation first... Something that seems to be missing here..    EDIT Doc

Doc


Thank you. Where did you find that documentation?
It looks like the doc I have not been able to find.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 17, 2013, 09:14 pm

Quote
What is the disadvantages of increasing the buffer size?

In my opinion, that allows for sloppy coding. There are situations where it is unavoidable, I guess. The biggest drawback is that there limited memory available, and there is only one value that defines the size of the incoming AND outgoing serial buffers.

Still, it may be necessary, in your case, to make the change.


I agree the coding is not prefered. But if the alternative is to copy buffer content into a variable for "level2-buffering", increased buffer sounds less bad.
That input and output buffer can't be set individually is a downside though.
Title: Re: GPS: set update frequency
Post by: mikalhart on Oct 18, 2013, 06:00 pm
Just a couple of observations that might be useful here:

Quote
If the encode() takes longer to process than the time between sentences that will still cause problems.

encode() is very fast in both TinyGPS and TinyGPS++.  Using a static simulated data feed, we've measured that either library can process hundreds of sentences per second without issue.

What's more likely is that delays in the main body of the code are clogging things up, although it's not clear that's even happening here.  Fortunately, it's easy to detect this condition by examining whether the TinyGPS[++] object is experiencing an increasing checksum failures.  Also, if you're using SoftwareSerial, look to see if you're experiencing any overflow().  If checksums are not increasing, then your sketch is processing the incoming stream perfectly well.  There should be no need to buffer anything.

Tip: never use delay() in a TinyGPS[++] sketch.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 19, 2013, 05:43 pm
About 100 sentences pr second, is an average of 10ms/sentence  sounds like what I've measured. My static test set, I think it was 7 sentences (5 supported + 2 unsupported, I don't have it on this pc) took 75ms to process. After modifying the library to skip the sentence if the first 5 bytes indicates an unsupported sentence, the same data set took 48ms. So clearly some (longer) sentences takes longer to process than others (shorter).

How long will it take for the buffer to overflow at 9600 baud? If that is more than the processing time for the longest sentence, there will be overflow.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 19, 2013, 06:02 pm
Quote
How long will it take for the buffer to overflow at 9600 baud?

That depends on how often/quickly data arrives. Once the last sentence in the batch arrives, there is a relatively long delay before another character arrives.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 19, 2013, 09:04 pm

Quote
How long will it take for the buffer to overflow at 9600 baud?

That depends on how often/quickly data arrives. Once the last sentence in the batch arrives, there is a relatively long delay before another character arrives.



Assumed buffer is empty and the gps start sending data continuously at max speed. How long will it take to fill the buffer?
Title: Re: GPS: set update frequency
Post by: wildbill on Oct 19, 2013, 09:30 pm
9600 baud is ~960 characters/second. The buffer on the UNO is 64 bytes, so you have about 67ms of continuous input before overflow if you're not reading.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 20, 2013, 02:55 am
Quote
so you have about 67ms of continuous input before overflow if you're not reading.

So, if parsing takes 48 milliseconds, you should never have a buffer overflow problem.

Serial.print()s take time, too, though. What exactly is happening in this 48 millisecond interval? Just the parsing or the parsing and the printing?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 21, 2013, 12:08 am
PaulS, you're spot on. Serial.print took almost half the processing time.

So my theory of why the data is delayed is wrong. Then, how can I figure out why there is such major delay in the position data?
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 21, 2013, 12:16 am
Quote
Then, how can I figure out why there is such major delay in the position data?

Each sentence includes a time-stamp. What we need to understand is where the discrepancy comes from. Are the sentences arriving late? That is, is there a difference between the time stamp in the sentence and the time when you get the sentence? Perhaps your clock is wrong? :)

How are you determining that there is a delay? You might have mentioned that already, but this thread has wandered all over the place.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 21, 2013, 12:35 pm
I notice the delay with this test:
Power on
Stand still until I the gps has connection with at least 10 satellites and position is stable.
Run about 50m.
Stop and wait for a few seconds.
Run 50m more in the same direction.
Stop and wait for gps position to update.

The printed positions shows exactly my pattern of stand still-run-stand still-run-stand still - with a delay of 15-20 seconds. I did not pay attention to the timestamp vs clock.
Title: Re: GPS: set update frequency
Post by: michinyon on Oct 21, 2013, 12:43 pm
GPS devices implement a whole bunch of funny strategies  to work well in cars.

if you are moving slowly,  they tend to assume that you are actually stationary,  because they don't want you to feel that you car is sliding halfway across the road when you are stopped at the traffic light.    That is,   until it realises that you really have moved,   in which case it updates suddenly to the new position.

On some GPS devices,   you can turn these annoying features off,    and see what it really has calculated.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 21, 2013, 12:57 pm

On some GPS devices,   you can turn these annoying features off,    and see what it really has calculated.


Which leads me back to; where the heck did Docedison find the doc he partially pasted into this post?


Doc
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 21, 2013, 01:01 pm
Along with what michinyon suggests, my GPS has two modes - walk and drive. Perhaps yours can be similarly configured. For a GPS in a car, 50 meters is not a significant movement. Still, even when driving, my GPS shows much better reaction than yours seems to.

I think that you do need to pay attention to the time stamps vs. clock. Does the last time stamp before you start moving correlate to your clock time? After you run 50 meters, does the next time stamp correlate with your clock? Is the position data in that sentence correct, or not?
Title: Re: GPS: set update frequency
Post by: petterg on Oct 21, 2013, 01:19 pm

Along with what michinyon suggests, my GPS has two modes - walk and drive. Perhaps yours can be similarly configured.

How did you do that configuration?
If I'm lucky my gps will understand the same commands.


I think that you do need to pay attention to the time stamps vs. clock.

I will, once the rain stops.
Title: Re: GPS: set update frequency
Post by: PaulS on Oct 21, 2013, 01:28 pm
Quote
If I'm lucky my gps will understand the same commands.

I set mine using the touch screen. Your GPS chip is only one part of a GPS. It's a critical component, but not the only one.

I suspect that the walk/drive option is used to tell the GPS chip how often to report data. But, before you try to go there, I think it's important to understand whether the delay is in the time stamp or the time it takes to recognize that your position has changed. If the time stamp updates correctly, but the corresponding position data is not in sync, you have one problem. If the timestamps are incorrect, you have a different problem. Knowing which you have is crucial to solving it.
Title: Re: GPS: set update frequency
Post by: alnath on Oct 21, 2013, 01:38 pm
Have you already seen thins link ?
http://tutorial.cytron.com.my/2012/08/16/skm53-gps-module-starter-kit-skgps-53/ (http://tutorial.cytron.com.my/2012/08/16/skm53-gps-module-starter-kit-skgps-53/)

it gives some commands to change your chip configuration ( baud rate, NMEA sentences allowed ...)
I didn't try, I don't use the SKM53 ;)
HTH
Title: Re: GPS: set update frequency
Post by: petterg on Oct 21, 2013, 08:39 pm

Have you already seen thins link ?
http://tutorial.cytron.com.my/2012/08/16/skm53-gps-module-starter-kit-skgps-53/ (http://tutorial.cytron.com.my/2012/08/16/skm53-gps-module-starter-kit-skgps-53/)

it gives some commands to change your chip configuration ( baud rate, NMEA sentences allowed ...)
I didn't try, I don't use the SKM53 ;)
HTH


I hadn't seen that exact link, just a similar one.
There surely is a set of commands that can be sent to the gps to change its configuration. The challenge is to figure out / find docs for how those commands are supposed to look. I did only see commands for changing baudrate in that link.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 21, 2013, 08:48 pm

Quote
If I'm lucky my gps will understand the same commands.

I set mine using the touch screen. Your GPS chip is only one part of a GPS. It's a critical component, but not the only one.

I suspect that the walk/drive option is used to tell the GPS chip how often to report data. But, before you try to go there, I think it's important to understand whether the delay is in the time stamp or the time it takes to recognize that your position has changed. If the time stamp updates correctly, but the corresponding position data is not in sync, you have one problem. If the timestamps are incorrect, you have a different problem. Knowing which you have is crucial to solving it.


I thought you were having a gps chip for an arduino project. Sorry, my bad.

I totally see your idea for timestamp debug. Still waiting for the rain to stop as I don't want to fill my laptop with water for this test.
Title: Re: GPS: set update frequency
Post by: petterg on Oct 22, 2013, 01:56 am
Finally a doc!
http://www.rigacci.org/wiki/lib/exe/fetch.php/doc/appunti/hardware/gps/mtk_nmea_packet_0.8_transystem.pdf

Probably not complete, and probably a bunch of commands this particular chip does not support, but its a doc!

Does anyone see a command in there that may be related to the driving vs walking position calculation that Michinyon and PaulS mention some posts ago?

Edit:
Seems to be a more complete version here:
https://www.olimex.com/Products/Modules/GPS/MOD-GPS/resources/MOD-GPS-MTK-NMEA.pdf
Title: Re: GPS: set update frequency
Post by: petterg on Oct 25, 2013, 11:29 pm
I'm trying to make the gps respond to the PMTK messages as listed in the doc, but I cant find any PMTK_ACK from the gps, and it's output doesn't change either.

Here's the code I'm using:
Code: [Select]

#include <TinyGPS++.h>

static const int GPSBaud = 9600;
int i = 0;
char parity = 0;
bool pmtkend = true;

// The TinyGPS++ object
TinyGPSPlus gps;

//Testmessage
  char *cmd1 = "PMTK000*";
//Enable/disable outputs
  char *cmd2 = "PMTK314,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0*";

void setup()
{
  Serial.begin(115200);
  Serial1.begin(GPSBaud);
delay(10000);
  Serial.println(F("GPS config"));
  Serial.println();
  while (*cmd1) sendPMTK(*cmd1++);
  while (*cmd2) sendPMTK(*cmd2++);
  Serial.println(F("/GPS config"));
}

void loop()
{
  while (Serial1.available() > 0 && i < 50000) {
    i++;
    Serial.print((char)Serial1.read());
  }
}

void sendPMTK(char c)
{
  if(pmtkend)
  {
    pmtkend = false;
    Serial1.print('$');
  }
 
  Serial1.print(c);
 
  if(c == '*')
  {
    Serial1.println(parity);
//    Serial1.print('\r');
//    Serial1.print('\n');
    parity = 0;
    pmtkend = true;
  }
  else
  {
    parity ^= c;
   
  }
}

void displayInfo()
{
  Serial.print(F("Location: "));
  if (gps.location.isValid())
  {
    Serial.print(gps.location.lat(), 6);
    Serial.print(F(","));
    Serial.print(gps.location.lng(), 6);
  }
  else
  {
    Serial.print(F("INVALID"));
  }

  Serial.print(F("  Date/Time: "));
  if (gps.date.isValid())
  {
    Serial.print(gps.date.month());
    Serial.print(F("/"));
    Serial.print(gps.date.day());
    Serial.print(F("/"));
    Serial.print(gps.date.year());
  }
  else
  {
    Serial.print(F("INVALID"));
  }

  Serial.print(F(" "));
  if (gps.time.isValid())
  {
    if (gps.time.hour() < 10) Serial.print(F("0"));
    Serial.print(gps.time.hour());
    Serial.print(F(":"));
    if (gps.time.minute() < 10) Serial.print(F("0"));
    Serial.print(gps.time.minute());
    Serial.print(F(":"));
    if (gps.time.second() < 10) Serial.print(F("0"));
    Serial.print(gps.time.second());
    Serial.print(F("."));
    if (gps.time.centisecond() < 10) Serial.print(F("0"));
    Serial.print(gps.time.centisecond());
  }
  else
  {
    Serial.print(F("INVALID"));
  }

  Serial.println();
}


Does anyone see any problems with the code?
(Changing "Serial1" to "Serial" in function sendPMTK makes the sent messages look correctly on computer)
Title: Re: GPS: set update frequency
Post by: petterg on Oct 30, 2013, 08:41 pm
I've gotten a tiny bit further on this: I found a forum where a guy starts a thread asking if anyone knows how to enable GLL messages from the SKM53. The thread ends with him telling he solved the problem - not telling how. At least then I know the chip accepts incoming commands.
(I've posted in the thread and sent pm to the guy, without any reply.)

I've also realized that if I walk really slowly, I can walk almost 30m without any change in the position output from the gps.
Also, if I put the gps in a car with a videocamera that records both the pc monitor and showing where I drive, the position output is just a second or so behind. This I think would confirm that there is a "car-mode" enabled in the chip.
(I did these tests running an arduino code that just outputs anything from the Serial1 buffer to Serial.)

I guess the only way to get going on this project is to figure out what kind of commands the SKM53 accepts.
Title: Re: GPS: set update frequency
Post by: petterg on Nov 17, 2013, 11:14 pm
Today I finally learned something! Even two things.

1) Using a Due (84MHz) is a major improvement over ProMicro (16MHz) when it comes to position delay. Even when not using any library - just passing the serial data directly trough to the pc.
Based on the information that has come out from this thread I have no idea why. Could it be that the gps is powered by 3,3V rather than 5V? Could there be some kind of buffer in the GPS that buffers position data, but not time? If so, why would the MPU speed have any effect on this as long as the baud rate is identical?

2) PMTK-messages work for configuring the skm53  - if checksum is sent correctly.
Say if the checksum is 0b 0100 1101 (4D) I sent those exact bits, but I should have sent 0b 0011 0100 0100 0100 (That is ascii '4' followed by ascii 'D'). After this discovery configuration seems to work with ProMicro, but not with Due - probably it's serial input requires 5V. (I'm surprised that it works on 3,3V at all.)
Title: Re: GPS: set update frequency
Post by: lar3ry on Nov 18, 2013, 05:23 am
For some reason, I missed this thread. Here's a little thing I whipped up to configure my SKM53. NOte the usage in the first comment, which tells you that you enter the line from $ to *, inclusive. The checksum will be calculated for you.

Code: [Select]

/*
Enter a packet starting with $PMTK, and ending with a *
In this sketch the * indicates the end of transmission.
The sketch takes the packet, adds a checksum and sends it to
the GPS.

Note: If you change the baud rate of the GPS, you will have to
close the Serial monitor, change the baud rate in Serial1
(or whatever Serial port you use) and restart it.

Set your Serial Monitor to No Line Endings. The sketch supplies those too,
and if you send them, they will get you out of sync.

This sketch assumes you are running an Arduino Mega2560, and
have a serial GPS on Serial1. Change this to suit your own
configuration. (Software Serial for Uno, etc.)

One packet type not mentioned in the document at
http://api.ning.com/files/L30xp1HTxtEUhhmBj4yYWIB31IDKs*xJNecJYvcEKZofuHJcZ3wnTMuZL5FeJC535I6DJbBZZE7FpJwnQPxgT9yKLCibZaZj/NMEA_Packet_Userl.pdf
is packet type 220. Here are a couple of samples, as you
would enter them into the Serial Monitor

$PMTK220,200*
$PMTK220,500*
$PMTK220,1000*
$PMTK220,2000*

These will set the reporting interval to 200, 500, 1000,
and 2000 milliseconds, or 5, 2, 1, .5 Hz,  respectively.
*/

char sentence[80];
int idx = 0;

void setup() {
 // initialize both serial ports:
 Serial.begin(57600);
 Serial1.begin(9600);
}

void loop() {
 // read from Serial1, send to Serial:
 if (Serial1.available()) {
   Serial.write(Serial1.read());
 }

 if (Serial.available())  {
   char inbyte = Serial.read();
   sentence[idx++] = inbyte;
   sentence[idx] = 0;
   if (inbyte == '*') {
     for (int i=0; i < strlen(sentence); i++) {
       Serial.print(sentence[i]);
     }
     Serial.println();
     sendConfig ();
     sentence[0] = 0;
     idx = 0;
   }
 }
}

void sendConfig () {
 unsigned char CS = 0;
 for (int i=1; i< strlen(sentence)-1; i++) {
   CS ^= sentence[i];
 }
 Serial1.print(sentence);
 if (CS < 10) Serial1.print("0");
 Serial1.print(CS,HEX);
 Serial1.println();
}
Title: Re: GPS: set update frequency
Post by: petterg on Nov 18, 2013, 02:44 pm
Thank you for that information on 220. Your code is much like the one I got working yesterday.
Title: Re: GPS: set update frequency
Post by: lar3ry on Nov 18, 2013, 05:51 pm
I wonder if youe slowness has anything to do with the library. I often take a critical look at what a library is doing for me, and sometimes find that there is only a small part of it I need. In those cases, I will sometimes write my own routine, or grab a chunk of code out of the library, and modify it.

If I get a chance later today, and if it warms up a bit, I'll try the walking experiment to see how far I have to walk to get a change.

I did check to see if it would go faster than 5 Hz, and yes, I managed to get it to 10Hz for sure, and I think it went o 20 Hz, but I couldn't be sure.
Title: Re: GPS: set update frequency
Post by: petterg on Nov 18, 2013, 11:48 pm
I would appreciate very much to know if you experience the same.

I can't blame the library as there is delay even when I forward the raw gps messages to pc.
There were significantly less delay when using the Due (3,3V/84MHz) than when using the ProMicro (5V/16MHz). I haven't got to test the effect of custom configuration of the gps, as when I finally got it to respond to the PMTK messages, it had started to rain, and it's been raining ever since.

I hope the delay will be minor after combining 220, 300, 301, 313 and 314-messages in some way., and make a vlotage shifter so config can be done from the Due.
Title: Re: GPS: set update frequency
Post by: lar3ry on Nov 19, 2013, 05:37 am
I tried to do the walking experiment, with the Arduino attached to my laptop. But it's dark, cold, and we have about 20 cm. of snow on the ground. It did seem to lag some, but I'll try again tomorrow, when I have time. I did a few calculations, and at my latitude, the fourth decimal place in latitude represents about 18.5 cm, and in longitude, about 12 cm.