My Code Writing Too Much Data Using Adafruit Ultimate GPS Logger

I’m using an Arduino Uno with an Adafruit Ultimate GPS Logger.

I am trying to log some basic GPS info along with voltage data from a few analog inputs onto the SD card on the shield. I need data at 5Hz, and need to have the data formatted as clean CSV to minimize downstream cleanup/processing. I started with the Adafruit sample code and modified it for my application.

The code seems to be working fine, with one exception: it is logging twice per cycle. For example, it gives me a full set of data at 55.2 sec, and a second set at 55.2, before moving onto 55.4 sec. I can’t figure out why and was hoping someone here could help. If there are other problems with the code, I’d be glad for additional feedback–this is all new for me.

The code:

#include <SPI.h>
#include <Adafruit_GPS.h>
#include <SoftwareSerial.h>
#include <SD.h>
#include <avr/sleep.h>

// Ladyada's logger modified by Bill Greiman to use the SdFat library
//
SoftwareSerial mySerial(8, 7);
Adafruit_GPS GPS(&mySerial);

// Set GPSECHO to 'false' to turn off echoing the GPS data to the Serial console
// Set to 'true' if you want to debug and listen to the raw GPS sentences
#define GPSECHO  false
/* set to true to only log to SD when GPS has a fix, for debugging, keep it false */
#define LOG_FIXONLY false  

int Li=A0; //Lf assigned to port A0
int Lf=A1;
int Lo=A2; 
int LiValue;
int LfValue;
int LoValue;

// this keeps track of whether we're using the interrupt
// off by default!
boolean usingInterrupt = false;
void useInterrupt(boolean); // Func prototype keeps Arduino 0023 happy

// Set the pins used
#define chipSelect 10
#define ledPin 13

File logfile;

// read a Hex value and return the decimal equivalent
uint8_t parseHex(char c) {
  if (c < '0')
    return 0;
  if (c <= '9')
    return c - '0';
  if (c < 'A')
    return 0;
  if (c <= 'F')
    return (c - 'A')+10;
}

// blink out an error code
void error(uint8_t errno) {
  /*
  if (SD.errorCode()) {
   putstring("SD error: ");
   Serial.print(card.errorCode(), HEX);
   Serial.print(',');
   Serial.println(card.errorData(), HEX);
   }
   */
  while(1) {
    uint8_t i;
    for (i=0; i<errno; i++) {
      digitalWrite(ledPin, HIGH);
      delay(100);
      digitalWrite(ledPin, LOW);
      delay(100);
    }
    for (i=errno; i<10; i++) {
      delay(200);
    }
  }
}

void setup() {

  pinMode(Li,INPUT);
  pinMode(Lf,INPUT);
  pinMode(Lo,INPUT);

  // connect at 115200 so we can read the GPS fast enough and echo without dropping chars
  // also spit it out
  Serial.begin(115200);
  Serial.println("\r\nUltimate GPSlogger Shield");
  pinMode(ledPin, OUTPUT);

  // make sure that the default chip select pin is set to
  // output, even if you don't use it:
  pinMode(10, OUTPUT);

  // see if the card is present and can be initialized:
    if (!SD.begin(chipSelect)) {      // if you're using an UNO, you can use this line instead
    Serial.println("Card init. failed!");
    error(2);
  }
  char filename[15];
  strcpy(filename, "GPSLOG00.CSV");
  for (uint8_t i = 0; i < 100; i++) {
    filename[6] = '0' + i/10;
    filename[7] = '0' + i%10;
    // create if does not exist, do not open existing, write, sync after write
    if (! SD.exists(filename)) {
      break;
    }
  }

  logfile = SD.open(filename, FILE_WRITE);
  if( ! logfile ) {
    Serial.print("Couldnt create "); 
    Serial.println(filename);
    error(3);
  }
  Serial.print("Writing to "); 
  Serial.println(filename);

  // connect to the GPS at the desired rate
  GPS.begin(9600);

  // uncomment this line to turn on RMC (recommended minimum) and GGA (fix data) including altitude
  GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCGGA);
  // uncomment this line to turn on only the "minimum recommended" data
  //GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);
  // Set the update rate
  GPS.sendCommand(PMTK_SET_NMEA_UPDATE_5HZ);   // 100 millihertz (once every 10 seconds), 1Hz or 5Hz update rate

  // Turn off updates on antenna status, if the firmware permits it
  GPS.sendCommand(PGCMD_NOANTENNA);

  // the nice thing about this code is you can have a timer0 interrupt go off
  // every 1 millisecond, and read data from the GPS for you. that makes the
  // loop code a heck of a lot easier!
  useInterrupt(true);

  Serial.println("Ready!");
}


// Interrupt is called once a millisecond, looks for any new GPS data, and stores it
SIGNAL(TIMER0_COMPA_vect) {
  char c = GPS.read();
  // if you want to debug, this is a good time to do it!
  #ifdef UDR0
      if (GPSECHO)
        if (c) UDR0 = c;  
      // writing direct to UDR0 is much much faster than Serial.print 
      // but only one character can be written at a time. 
  #endif
}

void useInterrupt(boolean v) {
  if (v) {
    // Timer0 is already used for millis() - we'll just interrupt somewhere
    // in the middle and call the "Compare A" function above
    OCR0A = 0xAF;
    TIMSK0 |= _BV(OCIE0A);
    usingInterrupt = true;
  } 
  else {
    // do not call the interrupt function COMPA anymore
    TIMSK0 &= ~_BV(OCIE0A);
    usingInterrupt = false;
  }
}

void loop() {
  if (! usingInterrupt) {
    // read data from the GPS in the 'main loop'
    char c = GPS.read();
    // if you want to debug, this is a good time to do it!
    //if (GPSECHO)
      //if (c) Serial.print(c);
  //}

    LiValue = analogRead(Li);
    LfValue = analogRead(Lf);
    LoValue = analogRead(Lo); 
  
  // if a sentence is received, we can check the checksum, parse it...
  if (GPS.newNMEAreceived()) {
    // a tricky thing here is if we print the NMEA sentence, or data
    // we end up not listening and catching other sentences! 
  
    // Don't call lastNMEA more than once between parse calls!  Calling lastNMEA 
    // will clear the received flag and can cause very subtle race conditions if
    // new data comes in before parse is called again.
    char *stringptr = GPS.lastNMEA();
    
    
    if (!GPS.parse(stringptr))   // this also sets the newNMEAreceived() flag to false
      return;  // we can fail to parse a sentence in which case we should just wait for another

    // Sentence parsed! 
    Serial.println("OK");
    if (LOG_FIXONLY && !GPS.fix) {
      Serial.print("No Fix");
      return;
    } 

    // Rad. lets log it!
    Serial.println("Log");

    logfile.print(GPS.hour, DEC); 
    logfile.print(":");
    logfile.print(GPS.minute, DEC); 
    logfile.print(':');
    logfile.print(GPS.seconds, DEC); 
    logfile.print('.');
    logfile.print(GPS.milliseconds);
    logfile.print(",");
    logfile.print(GPS.day, DEC); 
    logfile.print('/');
    logfile.print(GPS.month, DEC); 
    logfile.print(","); 
    logfile.print(GPS.latitude, 4); 
    logfile.print(", ");
    logfile.print(GPS.lat);
    logfile.print(", ");
    logfile.print(GPS.longitude, 4); 
    logfile.print(", ");
    logfile.print(GPS.lon);
    logfile.print(",");
    logfile.print(GPS.altitude); 
    logfile.print(",");
    logfile.print(LiValue);
    logfile.print(",");
    logfile.print(LfValue);
    logfile.print(",");
    logfile.println(LoValue);

   // uint8_t stringsize = strlen(stringptr);
   // if (stringsize != logfile.write((uint8_t *)stringptr, stringsize))    //write the string to the SD file
   //     error(4);
    if (strstr(stringptr, "RMC") || strstr(stringptr, "GGA"))   logfile.flush();
    Serial.println();
  }
}


/* End code */

Serial communications in an ISR is a no-no. Why are you using interrupts? It doesn't seem necessary for what you are doing.

My code uses the interrupts because it is based off the adafruit sample code. I don't have the expertise to question why they used it.

Can you explain why ISR and serial communication is a bad idea? Is there something about my application that makes that code a poor starting point?

Serial communications in an ISR is a no-no. Why are you using interrupts? It doesn’t seem necessary for what you are doing.

This is Adafruit’s work-around for two problems:

(1) The SD card can take too long to complete a write, and GPS characters are lost because of input buffer overflow;

(2) Many beginners try to print too much information from each GPS update. The Arduino eventually spends all its time trying to transmit each character.

This is particularly inefficient, because it doubles the number of times each character is copied, and it stores up to two complete NMEA sentences. :stuck_out_tongue: The Adafruit library uses the most RAM of any library (more comparisons here, here and here).

In general, handling received characters in the ISR is most efficient, because you avoid copying characters into and out of a buffer. But you have to be quick about it! This is the same guideline for any ISR, like a Pin Change Interrupt.

However, printing inside an ISR is generally frowned upon, because you should not wait in the ISR for the character to be sent.

NeoGPS has examples that show how the GPS stream can be parsed during the RX character interrupt, using one of the drop-in serial port replacements, NeoHWSerial, NeoSWSerial or NeoICSerial. They all have attachInterrupt methods that allow the sketch to handle each RX character during the interrupt. NeoGPS uses less than 1ms for an entire sentence of ~80 characters, so the average time spent in the RX char ISR is ~12us. Very lean! When the baud rate is 9600, the CPU utilization is ~1%.

But to the OP’s question:

it is logging twice per cycle. For example, it gives me a full set of data at 55.2 sec, and a second set at 55.2, before moving onto 55.4 sec. I can’t figure out why and was hoping someone here could help.

Because the Adafruit_GPS library is sentence-oriented, not update-oriented. It gives you the two sentences per update that you requested: RMC and GGA. This NeoGPS version gives you one complete fix per update:

#include <avr/sleep.h>

#include <SPI.h>
#include <SdFat.h>
#define CHIP_SELECT 10
#define LED_PIN 13
#define LOG_FIXONLY false
SdFat SD;

File logfile;

#include <NeoSWSerial.h>
NeoSWSerial gpsPort(8, 7); // AltSoftSerial on 8/9 would be better!

#include <NMEAGPS.h>
NMEAGPS gps;


int Li=A0; //Lf assigned to port A0
int Lf=A1;
int Lo=A2; 
int LiValue;
int LfValue;
int LoValue;


// read a Hex value and return the decimal equivalent
uint8_t parseHex(char c) {
  if (c < '0')
    return 0;
  if (c <= '9')
    return c - '0';
  if (c < 'A')
    return 0;
  if (c <= 'F')
    return (c - 'A')+10;
}

// blink out an error code
void error(uint8_t errno) {
  /*
  if (SD.errorCode()) {
   putstring("SD error: ");
   Serial.print(card.errorCode(), HEX);
   Serial.print(',');
   Serial.println(card.errorData(), HEX);
   }
   */
  while(1) {
    uint8_t i;
    for (i=0; i<errno; i++) {
      digitalWrite(LED_PIN, HIGH);
      delay(100);
      digitalWrite(LED_PIN, LOW);
      delay(100);
    }
    for (i=errno; i<10; i++) {
      delay(200);
    }
  }
}

void setup() {

  pinMode(Li,INPUT);
  pinMode(Lf,INPUT);
  pinMode(Lo,INPUT);

  // connect at 115200 so we can read the GPS fast enough and echo without dropping chars
  // also spit it out
  Serial.begin(115200);
  Serial.println( F("\r\nUltimate GPSlogger Shield with NeoGPS") ); // F macro saves RAM!
  pinMode(LED_PIN, OUTPUT);

  // make sure that the default chip select pin is set to
  // output, even if you don't use it:
  pinMode(10, OUTPUT);


  // see if the card is present and can be initialized:
  if (!SD.begin(CHIP_SELECT)) {      // if you're using an UNO, you can use this line instead
    Serial.println( F("Card init. failed!") );
    error(2);
  }

  char filename[15] = "GPSLOG00.CSV";
  for (uint8_t i = 0; i < 100; i++) {
    filename[6] = '0' + i/10;
    filename[7] = '0' + i%10;
    // create if does not exist, do not open existing, write, sync after write
    if (! SD.exists(filename)) {
      break;
    }
  }

  logfile = SD.open(filename, FILE_WRITE);
  if( ! logfile.isOpen() ) {
    Serial.print( F("Couldnt create ") ); 
    Serial.println(filename);
    error(3);
  }

  Serial.print( F("Writing to ") ); 
  Serial.println(filename);

  gpsPort.begin( 9600 );

  gps.send_P( &gpsPort, F("PMTK314,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0") ); // RMC_GGA
  gps.send_P( &gpsPort, F("PMTK220,200") );  // 5Hz
  gps.send_P( &gpsPort, F("PGCMD,33,0") );   // No antenna status messages needed

  Serial.println("Ready!");
}


void loop() {

  // Parse GPS characters until a complete fix has been assembled.
  if (gps.available( gpsPort )) {

    // A complete fix structure is finally ready, get it.
    gps_fix fix = gps.read();

    LiValue = analogRead(Li);
    LfValue = analogRead(Lf);
    LoValue = analogRead(Lo); 

    if (LOG_FIXONLY && !fix.valid.location) {
      Serial.print( F("No Fix") );
      return;
    } 

    // Rad. lets log it!
    Serial.println( F("Log") );

    logfile.print(fix.dateTime.hours); 
    logfile.print(':');
    logfile.print(fix.dateTime.minutes); 
    logfile.print(':');
    logfile.print(fix.dateTime.seconds); 
    logfile.print('.');
    if (fix.dateTime_cs < 10)
      logfile.print( '0' );
    logfile.print(fix.dateTime_cs);
    logfile.print(',');

    logfile.print(fix.dateTime.date); 
    logfile.print('/');
    logfile.print(fix.dateTime.month); 
    logfile.print(','); 
    logfile.print(fix.latitude(), 4); 
    logfile.print(", ");
    logfile.print(fix.longitude(), 4); 
    logfile.print(", ");
    logfile.print(fix.altitude() ); 
    logfile.print(',');
    logfile.print(LiValue);
    logfile.print(',');
    logfile.print(LfValue);
    logfile.print(',');
    logfile.println(LoValue);

    logfile.flush();
  }
}

It also uses the latest SdFat library, instead of the outdated SD library bundled with the IDE. SdFat has many bug fixes and speed improvements.

I’d like to think that it is easier to read, too. :slight_smile:

Your original version uses 22684 bytes of program space and 1625 bytes of RAM.
The NeoGPS version uses 19494 bytes of program space and 1076 bytes of RAM, a significant savings.

If you’d like to try it, NeoGPS and NeoSWSerial are available from the Arduino IDE Library Manager, under the menu Sketch → Include Library → Manage Libraries. BTW, you may still need to use Interrupt-Style processing to avoid losing GPS data during SD writes.

Cheers,
/dev

Thank you so much for your analysis and suggestions. I just tried your code and it seemed to work perfectly. I’m going to diveinto it now to make sure I understand what is going on, but this is a great start.

-dev:
It gives you the two sentences per update that you requested: RMC and GGA. This NeoGPS version gives you one complete fix per update:

Thought this might be the issue, but needed altitude from GGA and there doesn’t seem to be any way to get GGA without RMC. Again, thanks for the much more elegant solution.

Slash Dev, one follow up question:

The code is working great with the Adafruit Ultimater GPS Logger. I needed a second unit, and since the Adafruit was out of stock, ordered Sparkfun's GPS logger shield.

Do you know if there are specific differences between the shields that I need to address in the code? The problem I'm having is that the Sparkfun GPS data is logging at about 1Hz instead of 5Hz, or at least losing enough data to only log to the card about once per second.

Do you know if there are specific differences between the shields that I need to address in the code?

They both use the MediaTek MTK3339 GPS chip, so they should respond to the same commands.

Both shields have a switch to connect the GPS device to either pins 0/1 (Serial) or pins 8/9 (use AltSoftSerial). Your sketch was using pins 8 & 7, so I'm not sure the transmit pin is connected correctly.

I switched to AltSoftSerial on 8 and 9, but it is still logging data at 1Hz. I don't think it is a missing data issue; it is very consistent once a fix is acquired, but only at 1Hz.

I tried adding the *2C to the end of the 5Hz PMTK, tried 10Hz. All had the same result.

Any suggestions?

Hmm… Sometimes it needs delays between configuration commands. Try 100ms:

 gps.send_P( &gpsPort, F("PMTK314,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0") ); // RMC_GGA
  delay( 100 );
  gps.send_P( &gpsPort, F("PMTK220,200") );  // 5Hz
  delay( 100 );
  gps.send_P( &gpsPort, F("PGCMD,33,0") );   // No antenna status messages needed
  delay( 100 );

I think you should confirm that it is actually receiving commands. Try this:

#include <AltSoftSerial.h>
AltSoftSerial gpsPort; // pins 8/9

#include <NMEAGPS.h>
NMEAGPS gps;

void setup() {

  // connect at 115200 so we can read the GPS fast enough and echo without dropping chars
  // also spit it out
  Serial.begin(115200);

  gpsPort.begin( 9600 );

  gps.send_P( &gpsPort, F("PMTK314,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0") ); // RMC_GGA
  delay( 100 );
  gps.send_P( &gpsPort, F("PMTK220,200") );  // 5Hz
  delay( 100 );
  gps.send_P( &gpsPort, F("PGCMD,33,0") );   // No antenna status messages needed
  delay( 100 );

  Serial.println("Ready!");
}


void loop() {
  if (gpsPort.available())
    Serial.write( gpsPort.read() );
}

If you see sentences other than $GPGGA and $GPRMC, it is not receiving the commands. There must be some hardware issue, either with your Arduino pin 9, the shield connection or the switch.

If you see GGA and RMC only, but at 1Hz, it is not obeying the 5Hz command. You can also try requesting RMC only:

  gps.send_P( &gpsPort, F("PMTK314,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0") ); // RMC only

Some devices will not allow you to set an update rate that is too high for the baud rate. Try setting the GPS baud rate to 19200 or 38400:

  gps.send_P( &gpsPort, F("PMTK314,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0") ); // RMC_GGA
  delay( 100 );
  gps.send_P( &gpsPort, F("PGCMD,33,0") );   // No antenna status messages needed
  delay( 100 );
  gps.send_P( &gps_port, F("PMTK251,19200") );  // set baud rate
  gps_port.flush();                              // wait for the command to go out
  delay( 100 );                                  // wait for the GPS device to change speeds
  gps_port.end();                                // empty the input buffer, too
  gps_port.begin( 19200 );                      // use the new baud rate

  gps.send_P( &gps_port, F("PMTK220,200") );     // set the GPS update interval

The delay worked.

Thanks again!

Slash Dev

I'm trying to move this code to a Sparkfun Micro Pro and their mini GPS shield connected to a GP-735 to reduce the size of my project.

Any tips on what needs to change in the code with the Micro Pro or the 735? I started with the above code using altsoftserial on 8,9.

The GP- 735 is based on the ublox neo- 7 family of devices, so the commands have to change. Look at the spec and see if you can find similar commands in the proprietary NMEA commands.

It looks like the command to change the Hz is “measRate” as expressed in milliseconds. It’s page 129 of the spec you linked to.

I’m unsure whether implementing it would require changing the NeoGPS code, or just adding the correct command into my sketch. Any advice?

I’m unsure whether implementing it would require changing the NeoGPS code, or just adding the correct command into my sketch. Any advice?

No changes to NeoGPS required. There is an example program: ubloxRate.ino. It shows how to send these binary commands. Apparently, there is no NMEA text command to set the update rate.

I wouldn’t suggest trying to run ubloxRate.ino, as it requires reconfiguring NeoGPS to build the derived UBX classes. You can also configure the GPS in a way that it won’t listen or won’t talk, until you power it off and back on. :stuck_out_tongue:

Just grab some of the snippets. Get the command definitions, like

const unsigned char ubxRate5Hz[] PROGMEM =
  { 0x06,0x08,0x06,0x00,200,0x00,0x01,0x00,0x01,0x00 };

const char baud19200 [] PROGMEM = "PUBX,41,1,3,3,19200,0"; <-- I just made this for you

… and the routine that sends binary commands:

void sendUBX( const unsigned char *progmemBytes, size_t len )
{
  gpsPort.write( 0xB5 ); // SYNC1
  gpsPort.write( 0x62 ); // SYNC2

    ...

Then send the binary command the same way:

         sendUBX( ubxRate5Hz, sizeof(ubxRate5Hz) );

Just like your other program, you probably need to change the baud rate (an NMEA PUBX text command). You might be able to just send the command like this:

 gps.send_P( &gpsPort, (const __FlashStringHelper *) baud19200 );  <-- text command
  gpsPort.flush();
  gpsPort.end();

  delay( 500 );
  gpsPort.begin( 19200 );

If that doesn’t work, you may need to disable all the NMEA sentences first, then send the baud rate command, then enable the NMEA sentences you want. Notice the delays, too.