NeoGPS - configurable, ultra-small RAM footprint

/dev:
I just came across an interesting post about the awful state of the NMEA "standard"... from five years ago! It's called Why GPSes suck, and what to do about it.

An entertaining rant to read, thanks for that. It doesn't make life easier for the poor guy who just wants to set his clock made with an Arduino via GPS but at least it provides some comic relief.

What about SPI for communication between GPS and Arduino? SoftwareSerial is too slow

Both NeoGPS and TinyGPS just need to be fed bytes. The example programs all use Serial and SoftwareSerial, though.

You should be able to do the same thing with SPI as a source of bytes. It should be something like

void loop()
{
  spi.begin();
  while (something) {
    in_byte = spi.transfer( 0xFF );
    gps.decode( in_byte );
  }
  spi.end();
}

According to the ublox spec, section 4.4, 0xFF will be returned by the spi.transfer when there are no more available bytes. Although the decode methods will correctly ignore the 0xFF bytes, you probably want to watch for them so you can do something else. For TinyGPS, you will have to watch for consecutive 0xFF bytes:

void loop()
{
  uint8_t FF_bytes = 0;
  spi.begin();
  while (FF_bytes < 50) {
    in_byte = spi.transfer( 0xFF );
    gps.decode( in_byte );
    if (in_byte == 0xFF)
      FF_bytes++;
    else
      FF_bytes = 0;
  }
  spi.end();

  // Do something else
}

For NeoGPS, you can watch for a different return code:

void loop()
{
  spi.begin();
  do {
    byte = spi.transfer( 0xFF );
  } while (gps.decode(byte) != NMEAGPS::DECODE_CHR_INVALID);
  spi.end();

  // Do something else
}

This is just for reading bytes. If you need to write bytes (like a configuration command or a poll), you'll do the same kind of thing... the output bytes are written out by the same transfer method:

  spi.begin();
  while (something) {
    spi.transfer( out_byte );
  }
  spi.end();

What complicates this process is that the Neo-6 can send a byte back at the same time, if it has something. You have to continue decoding during a write:

  spi.begin();
  while (something) {
    in_byte = spi.transfer( out_byte );
    gps.decode( in_byte );
  }
  spi.end();

That's the basic idea, I think. You'll have to fill a lot blanks! :slight_smile: Keep us posted.

Cheers,
/dev

Thank you for your code! Nice job 8)

I just realized that many Ublox neo 6m boards don't implement SPI wires.. They only have VCC, GND, TX, RX .

:confused:

I think I need to find the correct output wires from the main processor and solder some custom wires

A recent discussion revealed that some GPS devices may report a message like "GNRMC" instead of the common "GPRMC". This led me down the rabbit hole of NMEA quasi-baz-standard... Oy!

It turns out that the first two characters are called the Talker ID. In this case, a GPS device was reporting "GPRMC" when only GPS satellites were used to calculate a fix, and "GNRMC" when a mix of GPS and GLONASS satellites were used. Furthermore, it could report "GLRMC" if only GLONASS satellites were used.

This variation also appears in the GSV sentences: GLGSV reports GLONASS satellite info, and GPGSV reports GPS satellite info. There are other Talker IDs, as well: GA (Galileo) and GB (BeiDou).

To accommodate these devices, I have updated NeoGPS to parse, save or skip the Talker ID.

While messing with this part of the parser, I also noticed that proprietary messages (those beginning with "$P") are also supposed to have a 3-character Manufacturer ID. For the ublox devices, these are the "$PUBX" messages. NeoGPS will now parse, save or skip the Mfr ID as well.

Suprisingly, this change increased speed by about 10%. It also eliminated the need to store the "GP" Talker ID, or any other talker ID, reducing the program space. In fact, ublox devices do not even have a proprietary message table. The first integer field after "$PUBX," identifies the proprietary message type: 0 or 4.

Woot.
/dev

One more iteration of squeezing better performance out of NeoGPS:

Configuration Sentence NeoGPS TinyGPS Performance
Improvement
Minimal GGA
RMC
436us
485us
- 70%
66%
DTL GGA
RMC
839us
859us
- 42%
40%
Nominal GGA
RMC
913us
936us
1448us
1435us
37%
35%
Full1 GGA
RMC
1094us
1075us
- 25%
25%

At less than 1ms to parse a sentence, that's about 0.1% CPU utilization, when the sentences are received once per second. Even with 10Hz updates, that's only 1% CPU utilization per sentence!

Cheers,
/dev


1 While the Full configuration is "only" 25% faster than TinyGPS, it handles more message types and fields. This configuration should probably be compared with TinyGPS++; the improvement would be much higher, as well as using 900 bytes less RAM.

A few updates:

  • A configuration error was fixed, which caused a compile error. :-[
  • Now supports Unos and other non-Mega boards. See NMEA.INO, other examples to be updated Real Soon Now.
  • Installation section is much more detailed.
  • Troubleshooting section includes general GPS device tips for connecting, baud rate and signal reception, as well as tips for resolving configuration errors.
  • Example configurations now include a Full configuration (everything!)
  • Example sketch shows how to convert UTC time to local time using a time zone offset, including partial hour time zones. See NMEA.INO
    I hope this makes NeoGPS more accessible. Looking under the hood may still be scary, but pre-defined configurations and Board-specific serial selection should make it usable by most programmers "out of the box".

If anyone has suggestions, especially for improving the documentation, I'd greatly appreciate it.

Cheers,
/dev

Updates:

  • Added explicit statements regarding GLONASS, BeiDou and Galileo support.
  • Extended the Troubleshooting section to include images and descriptions of the all-to-common input overflow (aka "trying to do too much").
  • Added a new section that shows which NMEA messages fill out which parts of a gps_fix. This helps immensely when Choosing Your Configuration.Cheers,
    /dev

Updates:

  • Added diagnostic program to auto-detect baud rate: NMEAdiagnostic
  • Added diagnostic program to display list of sentences emitted by your GPS device: NMEAorder
  • Increased quiet time window by 5ms or more by immediately keying off of last sentence in 1-second interval
  • Added test cases to NMEAtest
  • Added recommend sequence of programs to try
  • Updated docs, including comment blocks at top of each .INO
    Cheers,
    /dev

Updates:

  • Increased performance by 3%. A special-purpose divide routine decreased the lat/lon parse time by 25us.
  • Increased precision by using the minute's 6th digit after the decimal point. Overall significant digits for integer lat/lon is now about 10. For example, 117.0969485° longitude is parsed from 117° 5.816909". Integer form is 1170969485, parsed from NMEA field "11705.816909".
  • Fixed truncation error when longitude >= 128°. Test case added.
    Cheers,
    /dev

Updates:

  • Added an interrupt-driven example that uses the new NeoHWSerial replacement for HardwareSerial. The example registers a procedure to be called during the received character interrupt.
  • Worked around an IDE limitation on nested includes. Caused a "SoftwareSerial.h" not found on UNO platforms.
  • Added diagnostic messages to NMEA.ino to help the user determine that the baud rate is incorrect or that the sentence order is different from what was expected (different GPS devices have different sentence orders).Cheers,
    /dev

Updates:

  • All examples can now use NeoHWSerial, NeoICSerial or NeoSWSerial, in addition to the typical SoftwareSerial and AltSoftSerial. By simply including one of those headers, the example programs will use that library for the GPS device.
  • This allows all Arduino boards to use an interrupt-driven technique, as shown in the NMEA_isr.INO example.
    With this collection of serial drivers, I'd like to think the vast majority of Arduino boards and GPS device configurations are covered by the NeoGPS examples.

Cheers,
/dev

I'm working on a project that will require a GPS fusion with other instruments to overcome MEMS gyro drift in a fixed wing aircraft. Cheap is good and small is good. So at this point I'm using a Teensy 3.x and one of these GPS units that only has i2c. Since the processor is already too close to being saturated and TinyGPS++ is losing characters I decided that what I really need is to be able to program the GPS to get what I need, when I need it and nothing else. That seems to mean sending UBX commands.

After digging around I found neoGPS. It's quite the impressive project but at this point it won't build for me, throwing errors about being unable to find a serial port.

When I get home I'll look things over some more. But before I get too far into things are there any major obstacles (such as only having an I2c interface) controlling the NEO-6 with your library and the Teensy? At this point I'm just coming up to speed on the GPS hardware interfaces in general, thought I have worked with the iOS GPS libraries.

rockeronline00:
Thank you for your code! Nice job 8)

I just realized that many Ublox neo 6m boards don't implement SPI wires.. They only have VCC, GND, TX, RX .

:confused:

I think I need to find the correct output wires from the main processor and solder some custom wires

what I really need is to be able to program the GPS to get what I need, when I need it and nothing else. That seems to mean sending UBX commands.

Well, you might be ok sticking with NMEA if you turn off all sentences and then poll for the ones you want. You could even send some UBX binary configuration commands. Next, you could try polling for the more-compact PUBX NMEA messages instead of the typical GGA+RMC combo. If you still need more time or fewer interrupts, then I'd switch to the UBX binary protocol.

it won't build for me, throwing errors about being unable to find a serial port.

I've never built for the Teensy 3.0, but at least one other user has tried compiling NeoGPS on a 32-bit platform. I made a few changes related to its 32-bit quirks vs. the 8-bit AVRs.

The examples all try to pick a serial port, based on which Arduino you're using. I don't think it knows what to do for your build. Just delete the include of GPSport.h, and declare your own gps_port variable for it to use... assuming you have one for testing?

If you're using I2C in a polling style at this point, you'll need to modify the polling examples' loop to feed bytes to NeoGPS:

static void GPSloop()
{  
  while ( I2C_data_available() ) {
      .
      .
      .
    if (gps.decode( I2C_read_byte() ) == NMEAGPS::DECODE_COMPLETED) {

If you're using I2C in an interrupt style, you'll need to modify the NMEAfused_isr.ino example to hook it to your interrupt and pass it the byte:

static void GPSisr( uint8_t c )  // called when I2C has a byte?
{
  if (gps.decode( c ) == NMEAGPS::DECODE_COMPLETED) {
  .
  .
  .
}

are there any major obstacles (such as only having an I2c interface) controlling the NEO-6 with your library and the Teensy?

LOL, I don't know what I don't know. 32-bit vs 8-bit should be the biggest thing. If you can get the I2C bytes and pass them to NeoGPS' decode function, it won't care where they came from.

There could be some effort in getting the UBX protocol working over I2C. I think there's some Serial-specific timing related to the ACKs that are supposed to come back when you send configuration commands. And there's a sequence of steps you have to perform before you can even use the UBX POSNAV and VELNED messages.

I'd really suggest trying an optimized NMEA approach first. You may find the increased efficiency of NeoGPS is all you need. For the typical GGA+RMC, NeoGPS uses just 1.7ms vs 2.9ms per update (on a 20MHz AVR), nearly twice as fast. If you only need lat/lon/alt (not date/time/satellites/hdop), it's even faster.

So far, its seems that the biggest problem for new users is understanding when they need a safe copy of "the current fix". Understanding how new pieces of a fix should be merged as they are received is also important. If you're using the interrupt style, then gps.fix() could change at any time. You will have to double-buffer it.

Picking a custom configuration can also be tricky, but there is some compile-time assistance for invalid configurations.

Cheers,
/dev

Hi /dev,

thank you for your GPS library!
Also a really great documentation and examples that helped me understanding it.
But I have one question:
Is there a chance that you are going to implement a data member for the geoid-height field of the GGA-message?

Greetings,
enters

Is there a chance that you are going to implement a data member for the geoid-height field of the GGA-message?

Yes. All done! :slight_smile:

Cheers,
/dev

Great. Thank You very much!

Hi 1st thank you for this great work you have done, I am trying to do polling once every few minutes

I am struggling with ubx protocol few days now by doing polls and then parse bytes etc etc. just for 2-3 sentences I need like NAV_POSLLH, NAV_STATUS, NAV_TIMEUTC. It went quiet well for a beginner like me with some un-resolved issues that i did some tricks , not the best way but it seems to work.

Until i stepped into you library, I did not want to use the big footprint libraries thats why I went to ublox binary format. And I felt like re-inventing the wheel and said well lets try this, it looks promising.

So lets make it short after reading a lot and trying to understand as much as I can (am a beginner newbie) and following your instructions on the documentation step by step (https://github.com/SlashDevin/NeoGPS/blob/master/doc/ublox.md and https://github.com/SlashDevin/NeoGPS/blob/master/doc/Installing.md) tried to do the simple NMEA example with NeoSerial but some how I always get stuck here in compilation

I have done also a copy NMEAv2.ino and the same folder name NMEAv2 but also in the original file I get the exact same errors!

In file included from NMEAGPS.h:24:0,
from NMEAv2.ino:2:
GPSfix.h: In member function ‘gps_fix& gps_fix::operator|=(const gps_fix&)’:
GPSfix.h:442:19: error: ‘const struct gps_fix::valid_t’ has no member named ‘geoidHeight’
if (r.valid.geoidHeight)
^
GPSfix.h:443:9: error: ‘geoidHt’ was not declared in this scope
geoidHt = r.geoidHt;
^
GPSfix.h:443:21: error: ‘const class gps_fix’ has no member named ‘geoidHt’
geoidHt = r.geoidHt;

I tried to see with my reduced knowledge if there is any typo or something obvious but am banging my head into the wall :frowning:

could you give any hint or how to disable this I tried to disable it in GPSfix_gfg.h like //#define GPS_FIX_GEOID_HEIGHT

but still no success :frowning: any pointing to the right direction in a simple explanation for a beginner would be much appreciated

thank you in advance

Sorry, there was a copy and paste error for the new GEOID_HEIGHT feature. Fixed on github. Either get the whole zip again, or just get the one file that changed, GPSfix.h

Cheers,
/dev

Thank you for your immediate response, there is nothing to be sorry about, its still new and you have put a big effort on this and the rest of us are thank full for the work you are saving us.

Any way early adopters face this kind of issues :smiley:

UPDATE

Just in case anyone else faces this problem in the future:
PAY EXTRA attention from where the library is loaded (after I got the fix mentioned in previous developers post)
It was still not working SO after few times of reloading searching and replacing variables to find the error I came across the obvious the library was loaded from /usr/share/arduino/libraries/NeoGPS/ INSTEADS the location I had the sketch with the files as mentioned in the docs

SO never add the neoGPS to the libraries of arduino IDE OR at least if you do check FROM where your libraries are loaded !!! for the sketch you run (pay attention so they get loaded from within the sketch folder instead of the arduino libraries!!)

-------------------------ignore bellow this line the libraries where loadiang from wrong paths----------------------------------

although it was 1 line and first I fixed it in my copy and it did not work I got the whole new zip and started over from scratch. Still get this

In file included from /usr/share/arduino/libraries/NeoGPS/NMEAGPS.h:24:0,
from /usr/share/arduino/libraries/NeoGPS/NMEAGPS.cpp:19:
/usr/share/arduino/libraries/NeoGPS/GPSfix.h: In member function ‘gps_fix& gps_fix::operator|=(const gps_fix&)’:
/usr/share/arduino/libraries/NeoGPS/GPSfix.h:442:19: error: ‘const struct gps_fix::valid_t’ has no member named ‘geoidHeight’
if (r.valid.geoidHeight)
^
/usr/share/arduino/libraries/NeoGPS/GPSfix.h:443:9: error: ‘geoidHt’ was not declared in this scope
geoidHt = r.geoidHt;
^
/usr/share/arduino/libraries/NeoGPS/GPSfix.h:443:21: error: ‘const class gps_fix’ has no member named ‘geoidHt’
geoidHt = r.geoidHt;

so there must be something else missing somewhere else too :frowning:

hope the above helps to find it

thank you