Ultimate GPS @ 10Hz - logging with other signals - which library?

Hi all,

I have some technical questions in the pipeline relating to my attempted usage of an Adafruit Ultimate GPS shield with my Uno as part of a simple datalogging system, along with a bunch of other sensors.

But first, I would just like to put a general feeler for personal experiences out here:

  • Has anybody managed to successfully log GPS coords with the Ultimate shield at 10Hz [this is crucial for me] AND some other data signals too?

To elaborate on my issue:

Those who have worked with the Ultimate shield will I'm sure have played about with either the Adafruit_GPS library or the TinyGPS++ library. Unfortunately I'm struggling because neither library seems to be able to facilitate what I want.

Whilst TinyGPS seemed really promising; it had simple commands and it integrated into the script created for my other sensors really well, I could not figure out a way to read new GPS signals every 10Hz. The library only seemed capable of supporting the MTK chipset's standard GPS update rate (1Hz). I wonder if I have missed something here?

Anyway, this lead me onto attempts to use the Adafruit_GPS library instead. This option gave two sub-options for collecting GPS data from the module (as described in the 'parsing' example file of the library); 1. using interrupts every 1ms to write chars to a variable, then parsing when a new full NMEA sentence was recognised, or 2. reading chars straight from the GPS in the loop, then again parsing.

For Adafruit_GPS sub-option 1 (interrupts), I can do this at 9600 GPS baud (fine for 1Hz GPS updates), but not at 38400 (required to update at 10Hz instead of 1Hz). At 38400, the chars become garbled as soon as I get a signal fix. I wonder if this is an issue anyone else has experienced?

For Adafruit_GPS sub-option 2 (reading chars from loop), I simply cannot integrate this option into the script created for my other sensors; because this existing script uses the loop too. This option doesn't work because if you rely on the loop to read every single char from the GPS as it is transmitted (which must happen around every 1ms), if your loop takes longer than 1ms (mine does), you miss GPS chars, and hence can't put a single NMEA sentence together.

In summary, none of the options I believe I have at my disposable seem to be of use to achieve what I want. But I'm sure I must be missing something obvious, hence the post. I'm hoping there are others who have made use of the Ultimate Shield with other sensors using one of these libraries, so it would be great to hear from you if you have.

If feedback indicates that it is possible, I can post up some specific details of my efforts so far and try and tap into the wealth of technical knowledge here to find out why I'm failing.

Thanks for reading, look forward to hearing from you.

Yes, I've done this with an Adafruit Ultimate GPS logger, two sensors and a couple of analog reads. I found it impossible to get SoftwareSerial to work at 38400 baud so I wrote my own software serial code specifically for using a GPS with the Uno. It doesn't disable interrupts the way SoftwareSerial does, it runs at 9600, 19200 and 38400 baud and is pretty much a drop in replacement for SoftwareSerial. It only works with ASCII characters, which is what you're getting with a GPS. And you can't create a second object. You can find it here.

Another alternative is to use AltSoftSerial. It's more robust than what I did but it does require that you use pins 8 and 9 for RX and TX. That means you'll have to either add a jumper and sacrifice pin 7 as a duplicate, or cut a trace and solder a wire. AltSoftSerial also takes over timer 1, in case that's an issue for you.

The other thing you'll almost certainly have to do is add double buffering of whatever data you want to write to the SD card. This is necessary because the write latency will on occasion exceed the amount of time between GPS transmissions (100ms - receive and processing time). I'm pretty sure TinyGPS doesn't double buffer. I ended up writing my own GPS code too, but NeoGPS is very good library that I recommend. If you do some searching in the forum you'll find threads about it. Or if you just post a topic and write NeoGPS three times the author will, like Beetlejuice, appear and try to help you.

Hi JBoyton,

Thanks for the post - I recognise your username from the Adafruit forum where you’ve replied to one of my other threads regarding GPS issues.

I’ve updated that thread just this morning by the way, regarding the GPS baud update issues we discussed in my particular context, I’m keen to hear your thoughts on the results I found.

I will check out what you’ve attached here after work, as it certainly sounds more advanced than what I’ve been considering so far, many thanks for this.

A couple of immediate queries I have (forgive me if they can be answered in your attached project):

  • What sensors were you logging from alongside your Ultimate GPS and analog reads?
  • Did you have much support from Adafruit regarding the GPS baud update issues you came across? Would you / would they consider this a limitation of the hardware, or a limitation of the software/script chosen to utilise it?
  • Why would I need to use a jumper between pins 8 and 7? I am currently not utilising pins 8 and 9, so they are free to be used as you’ve described.

Thanks also for the comments on SD latency potentially causing data loss - it is something I’d already noticed and have on my list to improve. I had looked into an RTOS to overcome this (Nil or Free) but this path is a bit complex for my own current levels of experience and capability.

I will look into the double buffer idea you mentioned and into NeoGPS too. Thanks again for the suggestion.

The sensors in my logger are BMP183 and BMP280. I reworked the library for the '183 and got rid of the ~30ms of delays. This allowed me to overlap the sensor data request with the incoming GPS data, which at 38400 baud takes about 40ms (GGA and RMC sentences). The BMP280 runs in free mode so data is always available to be read.

I asked Adafruit about setting the baud rate but they didn't have any direct experience with it. I also emailed GlobalTop but, like their module sometimes does, they failed to respond to my request. I don't think it's an issue. The coin battery insures that the baud rate only has to be set to 38400 one time. And even if it failed I can see that on the OLED screen and simply reset the Uno so it can try again. I was being a little too thorough in trying to get it to work reliably from a cold start. It just kind of bugged me.

AltSoftSerial uses timer 1 hardware features to enable it to be more robust. For RX it uses the input capture mechanism and that's tied to digital pin 8 on an Uno. Fortunately the Ultimate GPS Logger shield uses pin 8 for RX. But AltSoftSerial also uses one of the output compare pins for TX, pin 9 on the Uno (pin 10 would also be possible). But on the shield TX is pin 7. So that means you'd have to do one of two things: (1) Jumper pin 7 to pin 9 and leave pin 7 unusable for other purposes; or (2) cut the trace on the shield between TX and pin 7 and solder a wire from TX to pin 9.

At 10Hz I don't think you need to get fancy. Typical SD card latency that I measured was under 2ms for writing a 512 byte block. But sometimes, unpredictably, the latency is much longer leading to dropouts. I found that a second buffer was enough. That way incoming GPS data could be processed before the SD card was finished writing the previous round of data. I did this by processing the GPS data on the fly, in the ISR with interrupts reenabled. The NeoGPS library can be configured to work the same way. An alternative is to increase the RX buffer to accommodate the GPS data. If you have the RAM that would be simpler. The Uno only has so much though.

I'd also recommend SdFat over SD. There's really no good reason to use SD. It's a wrapper for an older version of SdFat and hasn't been as well supported.

It's show time!

:slight_smile:

I have just one thing to add to jboyton's excellent comments. If you parse the GPS sentences (character bytes) into values (integers), there will be much less data to write to the SD. In fact, you can write just the pieces you need to analyze. All the NeoGPS examples write a CSV text format, which is easy to load in a spreadsheet or parse with another utility, like a Python script. NeoGPS can be configured to parse just the values (and sentences) you need, decreasing RAM usage, program space and CPU time.

If you are using some other NMEA-oriented tool to interpret the SD log files, though, you will need to log the raw characters. NeoGPS parsing would be of little use for that.

Cheers,
/dev

Thanks both, I very much appreciate the info. It’s been a long day for me, I think I’ll have to check out the NeoGPS library tomorrow with a clear head.

I’ve spent a couple of mins gathering some more detail related to the issue I described having with the Adafruit_GPS library. I was hoping in the mean time somebody may be able to give some suggestions as to how I might improve this option.

Please consider Case 1 and Case 2. Pics A and B depict the results of Case 1, pics C and D depict the results of Case 2.

Case 1 is some Adafruit_GPS library ‘parsing’ code. GPS baud is kept at 9600, GPS update rate is left as standard; 1 Hz. For Case 1 (as shown in pics A and B), using the Serial Monitor I can observe my GPS sentences being parsed and updating some variables, which are then displayed as called by the main loop.

Pic A shows the sentences being parsed successfully when there is no signal fix, and Pic B shows the sentence is still parsed correctly after I gain a signal fix and have some real data. Here is the code for Case 1:

// Test code for Adafruit GPS modules using MTK3329/MTK3339 driver
//
// This code shows how to listen to the GPS module in an interrupt
// which allows the program to have more 'freedom' - just parse
// when a new NMEA sentence is available! Then access data when
// desired.
//
// Tested and works great with the Adafruit Ultimate GPS module
// using MTK33x9 chipset
//    ------> http://www.adafruit.com/products/746
// Pick one up today at the Adafruit electronics shop 
// and help support open source hardware & software! -ada

#include <Adafruit_GPS.h>
#include <SoftwareSerial.h>
#include <SdFat.h>

// If you're using a GPS module:
// Connect the GPS Power pin to 5V
// Connect the GPS Ground pin to ground
// If using software serial (sketch example default):
//   Connect the GPS TX (transmit) pin to Digital 3
//   Connect the GPS RX (receive) pin to Digital 2
// If using hardware serial (e.g. Arduino Mega):
//   Connect the GPS TX (transmit) pin to Arduino RX1, RX2 or RX3
//   Connect the GPS RX (receive) pin to matching TX1, TX2 or TX3

// If you're using the Adafruit GPS shield, change 
// SoftwareSerial mySerial(3, 2); -> SoftwareSerial mySerial(8, 7);
// and make sure the switch is set to SoftSerial

// If using software serial, keep this line enabled
// (you can change the pin numbers to match your wiring):
SoftwareSerial mySerial(6, 5);

// If using hardware serial (e.g. Arduino Mega), comment out the
// above SoftwareSerial line, and enable this line instead
// (you can change the Serial number to match your wiring):

//HardwareSerial mySerial = Serial1;


Adafruit_GPS GPS(&mySerial);


// Set GPSECHO to 'false' to turn off echoing the GPS data to the Serial console
// Set to 'true' if you want to debug and listen to the raw GPS sentences. 
#define GPSECHO  true

// this keeps track of whether we're using the interrupt
// off by default!
boolean usingInterrupt = false;
void useInterrupt(boolean); // Func prototype keeps Arduino 0023 happy

int t,t1,t2=0;
SdFat SD;
File myFileGPS;
int x=1;

void setup()  
{
    
  // connect at 115200 so we can read the GPS fast enough and echo without dropping chars
  // also spit it out
  Serial.begin(115200);
  Serial.println("Adafruit GPS library basic test!");

  // 9600 NMEA is the default baud rate for Adafruit MTK GPS's- some use 4800
  GPS.begin(9600);
  //GPS.sendCommand("$PMTK251,38400*27<CR><LF>"); // Change GPS baud to 38400
        //GPS.sendCommand("$PMTK251,57600*2C"); // Change GPS baud to 57600
  //GPS.begin(38400);
  GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);  
  //GPS.sendCommand("$PMTK220,100*2F<CR><LF>"); // Change refresh rate to 10Hz
  
  // uncomment this line to turn on RMC (recommended minimum) and GGA (fix data) including altitude
  //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCGGA);
  // uncomment this line to turn on only the "minimum recommended" data
  //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);
  // For parsing data, we don't suggest using anything but either RMC only or RMC+GGA since
  // the parser doesn't care about other sentences at this time
  
  // Set the update rate
  //GPS.sendCommand(PMTK_SET_NMEA_UPDATE_1HZ);   // 1 Hz update rate
  // For the parsing code to work nicely and have time to sort thru the data, and
  // print it out we don't suggest using anything higher than 1 Hz

  // Request updates on antenna status, comment out to keep quiet
  //GPS.sendCommand(PGCMD_ANTENNA);

  // the nice thing about this code is you can have a timer0 interrupt go off
  // every 1 millisecond, and read data from the GPS for you. that makes the
  // loop code a heck of a lot easier!
  useInterrupt(true);

  delay(1000);
  // Ask for firmware version
  //mySerial.println(PMTK_Q_RELEASE);
  
     pinMode(10, OUTPUT);
   digitalWrite(10, HIGH);
   
  if (!SD.begin(4)) {
    Serial.println("SD initialization failed!");
    return;
  }
  Serial.println("SD initialization done.");
}


// Interrupt is called once a millisecond, looks for any new GPS data, and stores it
SIGNAL(TIMER0_COMPA_vect) {
  char c = GPS.read();
  // if you want to debug, this is a good time to do it!
#ifdef UDR0
  if (GPSECHO)
    if (c) UDR0 = c;  
    // writing direct to UDR0 is much much faster than Serial.print 
    // but only one character can be written at a time. 
#endif
}

void useInterrupt(boolean v) {
  if (v) {
    // Timer0 is already used for millis() - we'll just interrupt somewhere
    // in the middle and call the "Compare A" function above
    OCR0A = 0xAF;
    TIMSK0 |= _BV(OCIE0A);
    usingInterrupt = true;
  } else {
    // do not call the interrupt function COMPA anymore
    TIMSK0 &= ~_BV(OCIE0A);
    usingInterrupt = false;
  }
}

void loop()                     // run over and over again
{
  delay(100);
  Serial.print("\nLoop..");
    // read data from the GPS in the 'main loop'
    //char c = GPS.read();
  //Serial.print(GPS.newNMEAreceived());
  // if a sentence is received, we can check the checksum, parse it...
  if (GPS.newNMEAreceived()) {
    if (!GPS.parse(GPS.lastNMEA()))   // this also sets the newNMEAreceived() flag to false
      return;  // we can fail to parse a sentence in which case we should just wait for another    
    myFileGPS = SD.open("GPS.txt", FILE_WRITE);
    t2=millis();  
    t=t2-t1;
    t1=millis();
    Serial.print("\nTimestep(ms): ");Serial.print(t);
    Serial.print(", Location: ");
    Serial.print(GPS.latitude, 5); //Serial.print(GPS.lat);
    Serial.print(", "); 
    Serial.println(GPS.longitude, 5); //Serial.print(GPS.lon);
    myFileGPS.print(t);myFileGPS.print(",");
    myFileGPS.print(GPS.latitude, 5);myFileGPS.print(",");
    myFileGPS.print(GPS.longitude, 5);myFileGPS.println(",");  
    myFileGPS.close();
  }
}

Case 2 is almost the exact same script, except I have requested GPS baud to be upped to 38400. This is acknowledged and allowed by the chipset, as GPS.begin(38400) allows NMEA sentences to be parsed whilst waiting for a signal fix, as shown by Pic C.

But as soon as I get a signal fix in this case, for some reason the parsing no longer seems to be working, I get no meaningful sentences in Serial, as shown by Pic D. Also, my loop is no longer writing any lines to the SM, which tells me that full NMEA sentences are no longer being recognised after parsing.

So effectively, increasing the GPS baud has somehow resulted in a reduction in parsing capability, which makes no sense to me.

Does anyone have any thoughts on what might be happening here? I’d be grateful to hear them.

D.PNG

Code for Case 2 showing alterations made is here:

// Test code for Adafruit GPS modules using MTK3329/MTK3339 driver
//
// This code shows how to listen to the GPS module in an interrupt
// which allows the program to have more 'freedom' - just parse
// when a new NMEA sentence is available! Then access data when
// desired.
//
// Tested and works great with the Adafruit Ultimate GPS module
// using MTK33x9 chipset
//    ------> http://www.adafruit.com/products/746
// Pick one up today at the Adafruit electronics shop 
// and help support open source hardware & software! -ada

#include <Adafruit_GPS.h>
#include <SoftwareSerial.h>
#include <SdFat.h>

// If you're using a GPS module:
// Connect the GPS Power pin to 5V
// Connect the GPS Ground pin to ground
// If using software serial (sketch example default):
//   Connect the GPS TX (transmit) pin to Digital 3
//   Connect the GPS RX (receive) pin to Digital 2
// If using hardware serial (e.g. Arduino Mega):
//   Connect the GPS TX (transmit) pin to Arduino RX1, RX2 or RX3
//   Connect the GPS RX (receive) pin to matching TX1, TX2 or TX3

// If you're using the Adafruit GPS shield, change 
// SoftwareSerial mySerial(3, 2); -> SoftwareSerial mySerial(8, 7);
// and make sure the switch is set to SoftSerial

// If using software serial, keep this line enabled
// (you can change the pin numbers to match your wiring):
SoftwareSerial mySerial(6, 5);

// If using hardware serial (e.g. Arduino Mega), comment out the
// above SoftwareSerial line, and enable this line instead
// (you can change the Serial number to match your wiring):

//HardwareSerial mySerial = Serial1;


Adafruit_GPS GPS(&mySerial);


// Set GPSECHO to 'false' to turn off echoing the GPS data to the Serial console
// Set to 'true' if you want to debug and listen to the raw GPS sentences. 
#define GPSECHO  true

// this keeps track of whether we're using the interrupt
// off by default!
boolean usingInterrupt = false;
void useInterrupt(boolean); // Func prototype keeps Arduino 0023 happy

int t,t1,t2=0;
SdFat SD;
File myFileGPS;
int x=1;

void setup()  
{
    
  // connect at 115200 so we can read the GPS fast enough and echo without dropping chars
  // also spit it out
  Serial.begin(115200);
  Serial.println("Adafruit GPS library basic test!");

  // 9600 NMEA is the default baud rate for Adafruit MTK GPS's- some use 4800
  GPS.begin(9600);
  GPS.sendCommand("$PMTK251,38400*27<CR><LF>"); // Change GPS baud to 38400
        //GPS.sendCommand("$PMTK251,57600*2C"); // Change GPS baud to 57600
  GPS.begin(38400);
  GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);  
  //GPS.sendCommand("$PMTK220,100*2F<CR><LF>"); // Change refresh rate to 10Hz
  
  // uncomment this line to turn on RMC (recommended minimum) and GGA (fix data) including altitude
  //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCGGA);
  // uncomment this line to turn on only the "minimum recommended" data
  //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);
  // For parsing data, we don't suggest using anything but either RMC only or RMC+GGA since
  // the parser doesn't care about other sentences at this time
  
  // Set the update rate
  //GPS.sendCommand(PMTK_SET_NMEA_UPDATE_1HZ);   // 1 Hz update rate
  // For the parsing code to work nicely and have time to sort thru the data, and
  // print it out we don't suggest using anything higher than 1 Hz

  // Request updates on antenna status, comment out to keep quiet
  //GPS.sendCommand(PGCMD_ANTENNA);

  // the nice thing about this code is you can have a timer0 interrupt go off
  // every 1 millisecond, and read data from the GPS for you. that makes the
  // loop code a heck of a lot easier!
  useInterrupt(true);

  delay(1000);
  // Ask for firmware version
  //mySerial.println(PMTK_Q_RELEASE);
  
     pinMode(10, OUTPUT);
   digitalWrite(10, HIGH);
   
  if (!SD.begin(4)) {
    Serial.println("SD initialization failed!");
    return;
  }
  Serial.println("SD initialization done.");
}


// Interrupt is called once a millisecond, looks for any new GPS data, and stores it
SIGNAL(TIMER0_COMPA_vect) {
  char c = GPS.read();
  // if you want to debug, this is a good time to do it!
#ifdef UDR0
  if (GPSECHO)
    if (c) UDR0 = c;  
    // writing direct to UDR0 is much much faster than Serial.print 
    // but only one character can be written at a time. 
#endif
}

void useInterrupt(boolean v) {
  if (v) {
    // Timer0 is already used for millis() - we'll just interrupt somewhere
    // in the middle and call the "Compare A" function above
    OCR0A = 0xAF;
    TIMSK0 |= _BV(OCIE0A);
    usingInterrupt = true;
  } else {
    // do not call the interrupt function COMPA anymore
    TIMSK0 &= ~_BV(OCIE0A);
    usingInterrupt = false;
  }
}

void loop()                     // run over and over again
{
  delay(100);
  Serial.print("\nLoop..");
    // read data from the GPS in the 'main loop'
    //char c = GPS.read();
  //Serial.print(GPS.newNMEAreceived());
  // if a sentence is received, we can check the checksum, parse it...
  if (GPS.newNMEAreceived()) {
    if (!GPS.parse(GPS.lastNMEA()))   // this also sets the newNMEAreceived() flag to false
      return;  // we can fail to parse a sentence in which case we should just wait for another    
    myFileGPS = SD.open("GPS.txt", FILE_WRITE);
    t2=millis();  
    t=t2-t1;
    t1=millis();
    Serial.print("\nTimestep(ms): ");Serial.print(t);
    Serial.print(", Location: ");
    Serial.print(GPS.latitude, 5); //Serial.print(GPS.lat);
    Serial.print(", "); 
    Serial.println(GPS.longitude, 5); //Serial.print(GPS.lon);
    myFileGPS.print(t);myFileGPS.print(",");
    myFileGPS.print(GPS.latitude, 5);myFileGPS.print(",");
    myFileGPS.print(GPS.longitude, 5);myFileGPS.println(",");  
    myFileGPS.close();
  }
}

delay is not your friend when you're trying to keep up. 100ms is about 1k characters at 115200, or about 100 characters at 9600. You're losing a lot of data during that time.

Serial.print is not your friend when you're trying to keep up. After about 70 characters of printing, it will block while it waits for some of them to finally get printed. Just printing "Loop..." will eventually block (i.e., delay) for 8ms, causing the loss of 80 characters @ 115200, about two sentences worth.

The NeoGPS Troubleshooting section describes this common problem.

It looks like you're saving the parsed data, not the raw bytes, so I think NeoGPS could really help. It can be configured to parse only the lat/lon fields, which you can get from just one sentence. All the other sentences could be disabled.

Cheers,
/dev

I wouldn't know where to start with that. The Adafruit example sketch just barely works at 9600 baud and 1Hz. It's really not a good starting point, only a demonstration and an easy to test the hardware. I'm curious, how come your RX/TX aren't set to pins 8/7? Did Adafruit redesign the shield or are you doing something non-standard with it?

The 100ms delay is very defeating. You only have 100ms to do everything and you squander it right off the bat.

A simple example sketch that has a sturdier foundation would probably help you. I wish I had one to offer. Maybe /dev does.

By the way, I only just watched Beatlejuice for the first time a few months ago. I didn't even recognized Alec Baldwin at first.

jboyton:
I wish I had one to offer. Maybe /dev does.

Ok, I’ll offer this:

#include <Arduino.h>
#include <SoftwareSerial.h>
#include <SdFat.h>
#include "NMEAGPS.h"

static NMEAGPS gps;
SoftwareSerial mySerial(6, 5);

int    t,t1,t2=0;
SdFat  SD;
SdFile myFileGPS;

//------------------------------------

static void doSomeWork()
{
  if (gps.fix().valid.location) {
    t2=millis();  
    t=t2-t1;
    t1=millis();

    myFileGPS.print(t); myFileGPS.print(',');
    myFileGPS.print( gps.fix().latitude(), 5 );
    myFileGPS.print(',');
    myFileGPS.print( gps.fix().longitude(), 5 );
    myFileGPS.println(',');  

    // Clear out what we just printed.  If you need this data elsewhere,
    //   don't do this.
    gps.data_init();
  }

} // doSomeWork

//------------------------------------

static void GPSloop()
{  
  while (mySerial.available()) {
    if (gps.decode( mySerial.read() ) == NMEAGPS::DECODE_COMPLETED) {
      if (gps.nmeaMessage == NMEAGPS::NMEA_RMC)
        doSomeWork();
    }
  }
} // GPSloop

//--------------------------

void setup()
{
  Serial.begin( 115200 );
  Serial.println( F("NeoGPS library SD logging") );

  mySerial.begin( 9600 );

  // Turn off everything but RMC sentences
  gps.send_P( &mySerial, (str_P) F("$PMTK314,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0") );

  if (!SD.begin(4)) {
    Serial.println( F("SD initialization failed!") );
    return;
  }
  Serial.println( F("SD initialization done.") );
//  myFileGPS = SD.open( "GPS.txt", FILE_WRITE );
  myFileGPS.open( "GPS.txt", O_APPEND );
}

//--------------------------

void loop()
{
  GPSloop();
}

It compiles, but I really can’t test it. Also, I seem to have a different version of SDFat than you do. This compiles with the latest version (I think) from here.

Naturally, you will have to install NeoGPS (instructions here).

Cheers,
/dev

Hard coded pin numbers and no sync or file close? I’m so disappointed. :slight_smile:
I tried the sketch and it didn’t even create a file or complain about it when it failed.

I took the NeoGPS part of the code and inserted it into a 10Hz sketch I’d written a while back. But these example sketches don’t explain how to double buffer the data. At 10Hz, and especially with spending time reading other sensors, at some point the SD card latency will result in the situation where new GPS data will be arriving while the processor is still executing SdFat library code. If you don’t mind occasional data dropouts maybe you can live with it. Otherwise either the RX buffer has to be enlarged or another method employed to buffer the data.

For GGA and RMC sentences the RX buffer would have to be over 150 bytes. For either gSoftSerial or SoftwareSerial it is far more efficient to use a buffer size that is a power of 2. So that means a 256 byte RX buffer, a substantial chunk of the Uno RAM. AltSoftSerial is different in that regard so you could set it to 170 or whatever would cover the worst case. If you just need the RMC then I think the worst case is around 90 bytes or so.

I’m pretty sure there is another way to do this with NeoGPS that doesn’t require all of the buffer space, where the incoming GPS data is handled in the ISR.

For what it’s worth, here’s my single buffered example sketch with the NeoGPS stuff added.
It uses RX=8, TX=7 and SD_CS=10, as per the Ultimate GPS Logger shield.

// Uno + Ultimate GPS Logger
// GGA+RMC, 10Hz, 38400 baud

#define USE_NEO_GPS    true    // true=use NeoGPS; false=echo entire NMEA sentences to file

#include <gSoftSerial.h>
#include <SdFat.h>
#include "NMEAGPS.h"

#define RX_PIN         8
#define TX_PIN         7
#define SD_CHIP_SELECT 10
#define SYNC_PERIOD    5L*60L*1000L    // 5 minutes

gSoftSerial ss(RX_PIN, TX_PIN);
SdFat sdcard;
SdFile logfile;
static NMEAGPS gps;

long ms;
long prevMillis;

void setup()
{
  Serial.begin(115200);
  Serial.print(F("\nHello\n"));
  
  ss.begin(9600);
  ss.println(F("$PMTK251,38400*27"));  // 38400 baud
  ss.setBaudRate(38400);
  delay(100);
  ss.println(F("$PMTK314,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0*28"));     // RMC and GGA
  ss.println(F("$PMTK220,100*2F"));     // 10 Hz
  
  Serial.println(F("Type any char to start logging"));
  Serial.println(F("\nGPS output is displayed in the meantime\n"));
  delay(3000);
  while (!Serial.available()) {
    if (ss.available()) {
      char c = ss.read();
      Serial.write(c);
    }
  }
  while (Serial.available()) Serial.read();    // flush input buffer

  Serial.print(F("\n\nInitializing SD card..."));
  if (!sdcard.begin(SD_CHIP_SELECT, SPI_FULL_SPEED)) {
    Serial.println(F("Card init. failed!"));
    while (true) {}
  }
  Serial.println(F("done."));
  
  Serial.print(F("Opening file..."));
  char filename[15];
  strcpy(filename, "GPSLOG00.TXT");
  for (uint8_t i = 0; i < 100; i++) {
    filename[6] = '0' + i/10;
    filename[7] = '0' + i%10;
    if (! sdcard.exists(filename)) {
      break;
    }
  }

  if( ! logfile.open(filename, O_CREAT | O_WRITE | O_EXCL) ) {
    Serial.print("Couldnt create "); 
    Serial.println(filename);
    while (true) {}
  }
  Serial.print(" Writing to "); 
  Serial.println(filename);
    
  Serial.println("Type any character to close the file.");
  ms = SYNC_PERIOD;
  prevMillis = millis();
}


void loop()
{

#if USE_NEO_GPS
//
// Use NeoGPS to decode data and write to the log file
//
  while (ss.available()) {
    if (gps.decode( ss.read() ) == NMEAGPS::DECODE_COMPLETED) {
      if (gps.nmeaMessage == NMEAGPS::NMEA_RMC)
        doSomeWork();
    }
  }
 
#else
//
// Alternate: Simply echo entire NMEA sentences to the log file
//
  if (ss.available()) {
    char c = ss.read();
    logfile.write(c);
  }
#endif

  if (millis() > ms) {
    logfile.sync();
    ms += SYNC_PERIOD;  // 5 minutes;
  }
  
  if (Serial.available()) {
    logfile.close();
    Serial.println("\nFile closed.");
    while (1) {}
  }
}

static void doSomeWork()
{
    static uint16_t t, t1, t2;
  
    t2=millis(); 
    t=t2-t1;
    t1=millis();
  if (gps.fix().valid.location) {
    logfile.print(t); logfile.print(',');
    logfile.print( gps.fix().latitude(), 5 );
    logfile.print(',');
    logfile.print( gps.fix().longitude(), 5 );
    logfile.println(','); 

    // Clear out what we just printed.  If you need this data elsewhere,
    //   don't do this.
    gps.data_init();
  }
}

Hard coded pin numbers and no sync or file close? I'm so disappointed. :slight_smile:
I tried the sketch and it didn't even create a file or complain about it when it failed.

In the software industry, if it compiles, you can ship it. :smiley: Thanks for fleshing out that lame offering. All I could see was "Initialization failed".

I'm pretty sure there is another way to do this with NeoGPS that doesn't require all of the buffer space, where the incoming GPS data is handled in the ISR.

I think you just need extra instances of gps_fix, if I understand what you're asking:

const uint8_t MAX_FIX = 2; // 1 is how all the examples work
gps_fix  fix[ MAX_FIX ]; // buffered fixes
uint8_t  current_fix;
volatile uint8_t fixes;  // how many have been buffered

// Call this from the ISR
static void handleGPS( uint8_t received_byte )
{
  if (gps.decode( received_byte ) == NMEAGPS::DECODE_COMPLETED) {
    fix[ current_fix ] |= gps.fix(); // merge in this sentence's values

    if (gps.nmeaMessage == LAST_SENTENCE_IN_INTERVAL) {
      // Batch of sentences in this interval is complete,
      //   Save future fix data in the next available structure
      current_fix          = (current_fix+1) % MAX_FIX;
      fixes++;
    }
  }
}

void loop()
{
  if (fixes > 0) {
    uint8_t oldSREG = SREG;
    noInterrupts();
      uint8_t fix_to_write = (current_fix - fixes) % MAX_FIX; // the oldest fix
      fixes--;
    SREG = oldSREG;
    doSomeWork( fix_to_write );
  }

  if (millis() > ms) {
    logfile.sync();
    ms += SYNC_PERIOD;  // 5 minutes;
  }
  
  if (Serial.available()) {
    logfile.close();
    Serial.println("\nFile closed.");
    while (1) {}
  }
}

static void doSomeWork( uint8_t fix_to_write )
{
    static uint16_t t, t1, t2;
  
    t2=millis(); 
    t=t2-t1;
    t1=millis();

    gps_fix & one_fix = fix[ fix_to_write ];

    if (oneFix.valid.location) {
      logfile.print(t); logfile.print(',');
      logfile.print( oneFix.latitude(), 5 );
      logfile.print(',');
      logfile.print( oneFix.longitude(), 5 );
      logfile.println(','); 

      // Clear out what we just wrote so it can be reused in 'handleGPS'.
      one_fix.init();
    }
}

Odds are good that this is another lame offering... :frowning:

As far as size, the gps_fix structures are only as big as you have configured. In the OP's application, gps_fix structures are only 10 bytes each (just status and location). The gps object uses 18 bytes, for a total of 38 bytes when MAX_FIX = 2.

Cheers,
/dev

At 10Hz, and especially with spending time reading other sensors, at some point the SD card latency will result in the situation where new GPS data will be arriving while the processor is still executing SdFat library code.

I’m wondering if writing the parsed values, instead of every received character, would allow the SD card more time to finish the previous write. This may avoid the need to double-buffer the parsed values.

For example, a timestamp plus lat/lon should be < 16 bytes. In a 512-byte SD card buffer, that would be 32 intervals, or about 3 seconds at a 10Hz rate. As long as the SD card writes only occur in doSomeWork, the safest time to do so, I think the SD card should be able to finish processing the previous 512-byte write by the time another 512 bytes are ready… maybe?

Curious,
/dev

My understanding of SD cards is very limited. But they are not passive repositories of data. There's a processor in the card that is involved in attempting to keep a level wear on the flash. Sometimes the SD card simply takes longer to do a write, presumably because of housekeeping chores. There's also added latency when the library updates FAT (three or four 512 block transfers instead of just one).

I've seen writes sometimes take 90ms. I've read that they can take longer, depending on the card. When I set my logger to 10Hz it dropped data occasionally, unpredictably. I double buffered and it worked 100%.

I thought I'd seen an example you'd provided in the past with a call from the ISR and maybe that's what you reproduced above. It probably makes more sense to just increase the RX buffer. The Uno can be a problem with RAM availability though.

I thought I'd seen an example you'd provided in the past with a call from the ISR

Yes, it was probably the example from the Cosa version. I've been toying with providing a modified HardwareSerial that allows an ISR callback, defaulting to the current ::store_char. It would be a very limited modification, making it easy enough to keep in sync with new releases.

Cheers,
/dev

Thanks for the info chaps. It’s really good to read your discussion, I must admit I’ve struggled to take a lot of it in, although I am working through it all slowly.

I can see you are both big proponents of NeoGPS. I wonder if I could be a pain and back-track your attention to my Adafruit library efforts just for a bit longer.

The reasons I would like a bit more support with this option are:

  1. I can log @ 10Hz with this library when gathering chars from a loop. It frustrates me that I cannot make this work with interrupts/timers/counters too.
  2. I don’t yet fully understand the limitations that explain why this option currently isn’t working

To expand on point 1, please see this first script below, which instead of relying on interrupts to read the GPS as in my previous posts, it reads GPS RX data from the main loop:

// CA_GPS_Parsing_Loop

// CA take on Adafruit 'parsing' example code. Modified for 10Hz update with some code removed
// It relies on a 'loop' but cannot be feasible for CAKDAQ in this state. May have to consider interrupts, which will have its own complications..

// ---------------------------------------------------------------

#include <Adafruit_GPS.h>
#include <SoftwareSerial.h>
#include <SdFat.h>

int t,t1,t2=0;
SoftwareSerial mySerial(8, 7);
Adafruit_GPS GPS(&mySerial);
#define GPSECHO  true
boolean usingInterrupt = false;
void useInterrupt(boolean); // Func prototype keeps Arduino 0023 happy

// SD
SdFat SD;
File myFileGPS;
int x=1;

void setup()  
{    
  Serial.begin(115200);
  Serial.println("Adafruit GPS library basic test!");
  GPS.begin(9600);
  GPS.sendCommand("$PMTK251,38400*27\r\n"); // Change GPS baud to 38400
  //GPS.sendCommand("$PMTK251,57600*2C\r\n"); // Change GPS baud to 57600
  GPS.begin(38400);
  GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);  
  GPS.sendCommand("$PMTK220,100*2F\r\n"); // Change refresh rate to 10Hz
  // Request updates on antenna status, comment out to keep quiet
  GPS.sendCommand(PGCMD_ANTENNA);
  
    // SD
   pinMode(10, OUTPUT);
   digitalWrite(10, HIGH);
   
  if (!SD.begin(4)) {
    Serial.println("SD initialization failed!");
    return;
  }
  Serial.println("SD initialization done.");
}

//uint32_t timer = millis();
void loop()                     // run over and over again
{
    // read data from the GPS in the 'main loop'
    char c = GPS.read();
    //#ifdef UDR0
      //if (GPSECHO)
        //if (c) UDR0 = c;
    //#endif
    
  // if a sentence is received, we can check the checksum, parse it...
  if (GPS.newNMEAreceived()) {
      if (!GPS.parse(GPS.lastNMEA()))   // this also sets the newNMEAreceived() flag to false
        return;
      myFileGPS = SD.open("GPS.txt", FILE_WRITE);
      t2=millis();  
      t=t2-t1;
      t1=millis();
      Serial.print("\nTimestep(ms): ");Serial.print(t);
      Serial.print(", Location: ");
      Serial.print(GPS.latitude, 5); //Serial.print(GPS.lat);
      Serial.print(" "); 
      Serial.print(GPS.longitude, 5); //Serial.println(GPS.lon);
      myFileGPS.print(t);myFileGPS.print(",");
      myFileGPS.print(GPS.latitude, 5);myFileGPS.print(",");
      myFileGPS.print(GPS.longitude, 5);myFileGPS.println(",");  
      myFileGPS.close();
    //if (!GPS.parse(GPS.lastNMEA()))   // this also sets the newNMEAreceived() flag to false
      //return;  // we can fail to parse a sentence in which case we should just wait for another
  }
}

Results from Serial Monitor:

Timestep(ms): 85, Location: 0.00000 0.00000
Timestep(ms): 87, Location: 0.00000 0.00000
Timestep(ms): 89, Location: 0.00000 0.00000
Timestep(ms): 87, Location: 0.00000 0.00000
Timestep(ms): 86, Location: 0.00000 0.00000
Timestep(ms): 87, Location: 0.00000 0.00000
Timestep(ms): 86, Location: 0.00000 0.00000
Timestep(ms): 89, Location: 0.00000 0.00000
Timestep(ms): 89, Location: 0.00000 0.00000
Timestep(ms): 85, Location: 0.00000 0.00000
Timestep(ms): 89, Location: 0.00000 0.00000
Timestep(ms): 144, Location: 0.00000 0.00000
Timestep(ms): 54, Location: 0.00000 0.00000
Timestep(ms): 87, Location: 5217.78710 134.66149
Timestep(ms): 84, Location: 5217.78710 134.66149
Timestep(ms): 84, Location: 5217.78710 134.66140
Timestep(ms): 84, Location: 5217.78710 134.66140
Timestep(ms): 83, Location: 5217.78662 134.66140
Timestep(ms): 82, Location: 5217.78710 134.66140
Timestep(ms): 84, Location: 5217.78710 134.66149
Timestep(ms): 84, Location: 5217.78710 134.66149
Timestep(ms): 83, Location: 5217.78710 134.66149
Timestep(ms): 83, Location: 5217.78710 134.66159
Timestep(ms): 85, Location: 5217.78710 134.66169
Timestep(ms): 82, Location: 5217.78710 134.66169
Timestep(ms): 83, Location: 5217.78710 134.66169
Timestep(ms): 84, Location: 5217.78710 134.66180
Timestep(ms): 84, Location: 5217.78710 134.66180

As you can see from the SM results, new sentences are parsed successfully every ~80ms or so, which shows successful update to 10Hz, hence the GPS baud must have successfully updated to 38400 as requested.

[…to be continued, please see next post…]

…Now looking back at my ‘interrupts’ script (as in my previous posts), you can see I have made some changes as per your above suggestions. I’ve removed the delay() and Serial.print lines, UDR0 lines are included for debugging only:

// CA_GPS_Parsing_Interrupt_V1.2
//
// --------------------------------

#include <Adafruit_GPS.h>
#include <SoftwareSerial.h>
#include <SdFat.h>

SoftwareSerial mySerial(8, 7);

// If using hardware serial (e.g. Arduino Mega), comment out the
// above SoftwareSerial line, and enable this line instead
// (you can change the Serial number to match your wiring):

//HardwareSerial mySerial = Serial1;


Adafruit_GPS GPS(&mySerial);


// Set GPSECHO to 'false' to turn off echoing the GPS data to the Serial console
// Set to 'true' if you want to debug and listen to the raw GPS sentences. 
#define GPSECHO  true

// this keeps track of whether we're using the interrupt
// off by default!
boolean usingInterrupt = false;
void useInterrupt(boolean); // Func prototype keeps Arduino 0023 happy

int t,t1,t2=0;
SdFat SD;
File myFileGPS;
int x=1;

void setup()  
{
    
  // connect at 115200 so we can read the GPS fast enough and echo without dropping chars
  // also spit it out
  Serial.begin(115200);
  Serial.println("Adafruit GPS library basic test!");

  // 9600 NMEA is the default baud rate for Adafruit MTK GPS's- some use 4800
  GPS.begin(9600);

  GPS.sendCommand("$PMTK251,38400*27\r\n"); // Change GPS baud to 38400

  GPS.begin(38400);
  GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);  

  useInterrupt(true);

  delay(1000);
  // Ask for firmware version
  //mySerial.println(PMTK_Q_RELEASE);
  
     pinMode(10, OUTPUT);
   digitalWrite(10, HIGH);
   
  if (!SD.begin(4)) {
    Serial.println("SD initialization failed!");
    return;
  }
  Serial.println("SD initialization done.");
}


// Interrupt is called once a millisecond, looks for any new GPS data, and stores it
SIGNAL(TIMER0_COMPA_vect) {
  char c = GPS.read();
  // if you want to debug, this is a good time to do it!
#ifdef UDR0
  if (GPSECHO)
    if (c) UDR0 = c;  
    // writing direct to UDR0 is much much faster than Serial.print 
    // but only one character can be written at a time. 
#endif
}

void useInterrupt(boolean v) {
  if (v) {
    // Timer0 is already used for millis() - we'll just interrupt somewhere
    // in the middle and call the "Compare A" function above
    OCR0A = 0xAF; // 0xAF original (57 = 1/2) (2C = 1/4) (16 = 1/8) (A = 1/16)
    TIMSK0 |= _BV(OCIE0A);
    usingInterrupt = true;
  } else {
    // do not call the interrupt function COMPA anymore
    TIMSK0 &= ~_BV(OCIE0A);
    usingInterrupt = false;
  }
}

//uint32_t timer = millis();

void loop()                     // run over and over again
{
 
  if (GPS.newNMEAreceived()) {
    if (!GPS.parse(GPS.lastNMEA()))   // this also sets the newNMEAreceived() flag to false
      return;  // we can fail to parse a sentence in which case we should just wait for another
  //}
     
    myFileGPS = SD.open("GPS.txt", FILE_WRITE);

    myFileGPS.print(GPS.latitude, 5);myFileGPS.print(",");
    myFileGPS.print(GPS.longitude, 5);myFileGPS.println(",");  
    myFileGPS.close();
  }
}

Results from Serial Monitor:

$GPRMC,011538.094,V,,,,,0.00,0.00,060180,,,N*41
$GPRMC,011539.094,V,,,,,0.00,0.00,060180,,,N*40
$GPRMC,011540.094,V,,,,,0.00,0.00,060180,,,N*4E
$GPRMC,011541.094,V,,,,,0.00,0.0˜–˜›˜˜œ˜–––§•”F
$GPRMC,011542.094,V,,,,,0.00,0.00,060180,,,N*4C
$GPRMC,011543.094,V,,,,,0.00,0.00,060180,,,N*4D
$GPRMC,011543.304,V,,,,,0.00,0.00,060180,,,N*47
$GPRMC,011544.304,V,,,,,0.00,0.00,060180,,,N*40
$GPRMC,011545.304,V,,,,,0.00,0.00,060180,,,N*41
$GPRMC,011546.304,V,,,,,0.00,0.00,060180,,,N*42
Š$GPRMC,011547.304,V,,,,,0.00,0.00,060180,,,N*43
$GPRMC,011548.304,V,,,,,0.00,0.00,060180,,,N*4C
$GPRMC,011549.095,V,,,,,0.00,0.00,060180,,,N*46
$GPRMC,011549.314,V,,,,,0.00,0.00,060180,,,N*4C
Š$GPRMC,011550.314,V,,,,,0.00,0.00,060180,,,N*44
…$GPRMC,011551.314,V,,,,,0.00,0.00,060180,,,N*45
$GPRMC,011552.314,V,,,,,0.00,0.00,060180,,,N*46
$GPRMC,011553.314,V,,,,,0.00,0.00,060180,,,N*47
$GPRMC,011554.314,V,,,,,0.00,0.00,060180,,,N*40
$GPRMC,011555.09MK‰‰‰‰ r‚‚b‚r‚‚±ÁÅÅÅÅÕ±±±9Thbÿ$GPRMC,011555.31š–«–––––˜—˜˜œ0.00,011115,,,N*$GPRMC,011556.31š–«–––––˜—˜˜Œ0.00,011115,,,N*48
$GPRMC,011557.31š–«–––––˜—˜˜”0.00,011115,,,N*49
$GPRMC,011558.31š–«–––––˜—˜˜Œ0.00,011115,,,N*46
$GPRMC,011559.31š–«–––––˜—˜˜Œ0.00,011115,,,N*47
$GPRMC,011600.31š–«–––––˜—˜˜œ0.00,011115,,,N*48
$GPRMC,011601.09MK‰‰‰‰ r‚Âb’šŠråͱÁÅÅÅÅÕ²âbrThE5)ÿ$GPRMC,011602.09MKPKMSÓrºÂʺbrXÁÁÅÍѹÙÙÍű],0.3NK¦ ÉiűŠŠŠŠªbb±©Ý 5)ÿ$GPRMC,011604.00˜–PKMLL“éÂʒbrXÁÁÅÍѹÙÙÉÁ±]X`\be±ÍÍɹÝÕ±ÁÅÅÅÅÕ¢bb
Tn$GPRMC,011605.00˜–PKMSӓtÂʊbrX`aÅÍѹÙÙÉÁ±],0.20,350.69,011115,,,A*7$GPRMC,011606.00˜–(¥&¦¦ÉéÂʊbrXÁÁÅÍѹÙÙÅݱ]X`\fɱ͹ÕͱÁÅÅÅÅÕ²bb
Tn„$GPRMC,011607.00˜–PKMLLrºÂʒbrXÁÁÅÍѹÙÙÅÙ±],0.26,319.64,011115,,,A*7$GPRMC,011608.00˜–PKMLL“éÂʒbrXaÁÅÍѹÙÙÅÕ±],0.23,335.80,011115,,,A*$GPRMC,011609.000,A,5217.7891,N,˜˜¦&ÉɲŠšbºX`\bÕ±ÍÅá¹ÙѱÁÅÅʊªbbb
Tn$GPRMC,011610.00˜–(¥&¦¦ÉéÂʊbrXÁÁÅÍѹÙÙÅͱ]X`\`Q±ÍÁ͹áá±ÁÅÅÅÅÕ²bb
Tnp$GPRMC,011611.00˜–(¥&SӓtÂʊbrXÁÁÅÍѹÙÙÅͱ]X`\`U±ÍÁݹÅɱÁÅÅÅÅÕ²bb
TnŒ$GPRMC,011612.00˜–PKMLL“éÂʂbrX`aEÍѹÙÙÅɱ],0.0š–™™š—˜™–˜˜‘115,,,A*7$GPRMC,011613.00˜–(¥&SӓtÂʂbrXÁÁÅÍѹÙÙÅɱ]X`\bE±ÍÕÁ¹áɱÁÅÅÅÅÕ²bb
Tnf$GPRMC,011614.00˜–(¥&¦¦ÉéÂʂbrX``eÍѹÙÙÅ

Even now, my interrupt script results show garbled junk only as soon as a GPS fix is close to being seen, and this behaviour only occurs when a GPSbaud38400 request is made as shown (haven’t even bothered requesting for 10Hz updates yet). It’s worth bearing in mind that this print out is raw data directly from the GPS unit (my UDR0 lines) and so (as I understand) cannot be related any kind of parsing issue.

Doesn’t this indicate some issue related to the frequency of my interrupts? As standard, I believe the ISR designed here reads the GPS RX pin every 1ms. This seems to work for GPSbaud9600, if the baud were to be upped to 38400, wouldn’t it be prudent to call interrupts more frequently? Couldn’t this be a potential cause of my issue?

I can see you are a big proponent of NeoGPS. I wonder if I could be a pain and back-track your attention to my Adafruit library efforts just for a bit longer.

Well, I'm a proponent because I wrote it. :slight_smile: But I wrote it because I ran into the same problem that you, and many other people, are running into. Other libraries, like Adafruit's, are not structured very well when it comes to doing anything besides reading GPS characters. The examples are just too fragile, which you have confirmed:

  1. I can log @ 10Hz with this library when gathering chars from a loop. It frustrates me that I cannot make this work with interrupts/timers/counters too.

You are trying to modify the program, which is fine, but... it breaks the example program. Definitely frustrating.

  1. I don't yet fully understand the limitations that explain why this option currently isn't working

I can try to explain the limitations:

  1. SoftwareSerial will almost certainly not work at 38400. See Reply #2. This library is a fairly expensive way to do serial communications. Also, it can block other interrupts for significant periods of time, which usually breaks something else.

  2. Using Timer1 interrupts to read data that came in on a serial interrupt burdens the processor even more. At 9600 baud, it handles about one character per tick. At the attempted 38400, there could be 4 characters waiting. While they are handled in the Timer1 routine, no other interrupts can occur... interrupts like receiving GPS characters on mySerial. :frowning: SoftwareSerial must catch all 11 bits of each character. If interrupts are blocked for too long, the first bit(s) are lost. A '5' character, which is 01000101, is received as 01011110 or something, a '^'. The last few bits are garbage, because it lost the first few bits and it always reads 11 bits.

  3. Serial is at a lower baud rate than the GPS serial port, mySerial. It is a bottleneck. The GPS device is sending more data than you can possibly echo to Serial. Eventually, Serial blocks anything else from happening (like reading mySerial) while debug messages are printed. This is the "Trying to do too much" section in the NeoGPS Troubleshooting section. Garbled NMEA messages are the symptom of losing mySerial characters.

  4. The Adafruit library buffers the sentence as it is received, until you call parse. This means that the incoming data passes through two different buffers: the mySerial input buffer (64 bytes) and the Adafruit sentence buffer (120 bytes). Then the parse function walks through the buffer, which is the 3rd time a character has been accessed. Ain't nobody got time for dat! Our ISR musings are about how to avoid the input buffer as well -- why not parse the character when it is recevied?

  5. You are opening and closing the SD file on every sentence. That's too time-consuming. You must use something like jboyton's approach: open it once, sync it periodically, and close it when you know you're done with it. This will avoid strange truncations of the file.

  6. Many of these operations are time-consuming when compared to the received data rate. The example programs do not try to spread the processing out, to avoid bottlenecks. Because you have several things that can constructively or destructively interfere, it can look like random failures.

Soooo.... it's understandable that you want it to work, especially if you think you're tantalizingly close. The program just isn't structured properly, and it's doubtful that any amount of fiddling will make it do what you want, reliably.

Cheers,
/dev

CWAnthony:
Doesn't this indicate some issue related to the frequency of my interrupts? As standard, I believe the ISR designed here reads the GPS RX pin every 1ms. This seems to work for GPSbaud9600, if the baud were to be upped to 38400, wouldn't it be prudent to call interrupts more frequently? Couldn't this be a potential cause of my issue?

You're right about that. The timer 0 compare A interrupt isn't reading the RX buffer as frequently as the characters are coming in. So the RX buffer fills faster than it can be emptied. When you don't have a fix an RMC string is about 50 characters long so it all fits in the SoftwareSerial RX buffer which is 64 bytes by default. When you get a fix RMC is at least 70 bytes long. So it will overflow the buffer if you don't read it actively.

You'd think that since you're reading at 1/4 the rate the characters are coming in you'd still be able to keep it from overflowing. But the problem is that SoftwareSerial disables interrupts for over 95% of the time data is coming in and has very little tolerance for other interrupts. You can see that what's in the buffer is getting corrupted sometimes even before you got a fix.

If you replaced SoftwareSerial with one of the other options I mentioned you'd have a fighting chance. Unfortunately, Adafruit designed their example library in such a way that it is bound tightly to SoftwareSerial (or Hardware Serial). Unlike NeoGPS or TinyGPS you can't simply replace one serial input library with another. You'd have to modify the library.

If you can get it to work somehow more power to you. But I don't think it's worth it.

I could give you some of my code. I have a simple script that logs GGA and RMC strings at 10Hz without errors. It's double buffered. And it's interrupt driven so you can do other things while the data is coming in from the GPS. The main part of the loop looks like this:

void loop()
{
  while (!gps.gotRMC())
  {
    // do other stuff while data is coming in and being parsed
    // read sensors, etc.
  }
  logGpsValues();       // write selected fields to the log file
  gps.clear();          // release buffered data
  

  if (millis() > ms) {
    logfile.sync();
    ms += SYNC_PERIOD;  // 5 minutes;
  }
 
  if (Serial.available()) {
    logfile.close();
    Serial.println("\nFile closed.");
    while (1) {}
  }
}



void logGpsValues() {
  gps.parseDateTime();
  logfile.print(gps.timeStr());
  logfile.write(' ');
  logfile.print(gps.dateStr());
  logfile.write(', '); logfile.print(gps.latitude());
  logfile.write(', '); logfile.print(gps.longitude());
  logfile.print(", mode="); logfile.write(gps.modeChr());
  logfile.print(", hdop="); logfile.print(gps.hdop());
  logfile.print(", sats="); logfile.print(gps.satellites());
  logfile.print(", alt="); logfile.print(gps.altitude());
  logfile.print(", speed="); logfile.print(gps.speed());
  logfile.print(", course="); logfile.print(gps.course());
  logfile.println();
}

It uses my own library. The difference between it and /dev's, aside from the fact that he can write software way better than me and he'll work hard to support it, is that the GPS fields are not translated into integers or floats or whatever. They are stored as text. I did it that way because I wanted to log the data to a text file, not do any number crunching with my Uno.

You're welcome to have the entire script and the two library files. But I'm not sure you really want that.