Logger Serial1 data to SD card - Need help with stable logging

I got a

  • Arduino Mega 2560
  • Adafruit Ultimate GPS Breakout v3
  • Adafruit Micro-SD Breakout Board+

I have been working on a code to store GPS data in 2 files, but having problem saving all data from the NMEA.
Its looks like it jumps random at the serial data recived from the GPS.

The RAW data from the GPS is RAWLOG00.TXT and the normal data for tracking is GPSLOG00.CSV.

Can any one help me make the code work stable with all data from the GPS?

Here is the link to the code

kimlorentz.com/arduino/data/GPShard_and_soft_v4.ino

Looks like the data jumps?

SD is DOS FAT system. You have to close a file to update the FAT (file allocation table) or lose everything since the last file open. Maybe your code does this, Write a line, close the file, open the file, repeat.

If you put a line in setup() that adds a first line saying the program just started, then close and reopen the file, if the program does crash and restart, you will know.

I see big libraries and a lot of constants printed that are not flash-macro'd, so I wonder and if it was me I would find ways to check for crashes.

Not flash macro'd, the text is kept in RAM after copying from flash:

    Serial.println("Start logging");

Flash macro'd, the text is kept in and printed from flash:

    Serial.println( F( "Start logging" ));

I've got the code to work now but will make major changes to it.
I will now that it only logs once every hour. It should only log RMC and GGA.
I see that "#include <Adafruit_GPS.h>" is not necessary to use the GPS device. I can write commands to it manually.

This is how it wil work:
Step 1 - Turn on GPS
Step 2 - Get fix
Step 3 - Save the RMC and GGA file data to SD card
Step 4 - Set the GPS to standby
Step 5 - Wait one hour and perform other tasks in the meantime.
Step 6 - Start from Step 1 and so on.

Is it any string, buffer codes I can use to find the RMC and GGA data and make it store it?

Found one page that has a new whay to store data.
http://www.toptechboy.com/arduino/lesson-22-build-an-arduino-gps-tracker/

Will try this out later today

Here is his code.

#include <SD.h> //Load SD card library
#include<SPI.h> //Load SPI Library
 
#include <Adafruit_GPS.h>    //Install the adafruit GPS library
#include <SoftwareSerial.h> //Load the Software Serial library
SoftwareSerial mySerial(3,2); //Initialize the Software Serial port
Adafruit_GPS GPS(&mySerial); //Create the GPS Object
 
String NMEA1; //Variable for first NMEA sentence
String NMEA2; //Variable for second NMEA sentence
char c; //to read characters coming from the GPS
 
int chipSelect = 4; //chipSelect pin for the SD card Reader
File mySensorData; //Data object you will write your sesnor data to
 
 
void setup() {
  
  Serial.begin(115200); //Turn on serial monitor
  GPS.begin(9600); //Turn on GPS at 9600 baud
  GPS.sendCommand("$PGCMD,33,0*6D");  //Turn off antenna update nuisance data
  GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCGGA); //Request RMC and GGA Sentences only
  GPS.sendCommand(PMTK_SET_NMEA_UPDATE_1HZ); //Set update rate to 1 hz
  delay(1000);
  
  pinMode(10, OUTPUT); //Must declare 10 an output and reserve it to keep SD card happy
  SD.begin(chipSelect); //Initialize the SD card reader
  
  if (SD.exists("NMEA.txt")) { //Delete old data files to start fresh
    SD.remove("NMEA.txt");
  }
  if (SD.exists("GPSData.txt")) { //Delete old data files to start fresh
    SD.remove("GPSData.txt");
  }
 
}
 
void loop() {
  
  readGPS();
 
  if(GPS.fix==1) { //Only save data if we have a fix
  mySensorData = SD.open("NMEA.txt", FILE_WRITE); //Open file on SD card for writing
  mySensorData.println(NMEA1); //Write first NMEA to SD card
  mySensorData.println(NMEA2); //Write Second NMEA to SD card
  mySensorData.close();  //Close the file
  mySensorData = SD.open("GPSData.txt", FILE_WRITE);
  mySensorData.print(GPS.latitude,4); //Write measured latitude to file
  mySensorData.print(GPS.lat); //Which hemisphere N or S
  mySensorData.print(",");
  mySensorData.print(GPS.longitude,4); //Write measured longitude to file
  mySensorData.print(GPS.lon); //Which Hemisphere E or W
  mySensorData.print(",");
  mySensorData.println(GPS.altitude);
  mySensorData.close();
  }
  
}
 
void readGPS() {
  
  clearGPS();
  while(!GPS.newNMEAreceived()) { //Loop until you have a good NMEA sentence
    c=GPS.read();
  }
  GPS.parse(GPS.lastNMEA()); //Parse that last good NMEA sentence
  NMEA1=GPS.lastNMEA();
  
   while(!GPS.newNMEAreceived()) { //Loop until you have a good NMEA sentence
    c=GPS.read();
  }
  GPS.parse(GPS.lastNMEA()); //Parse that last good NMEA sentence
  NMEA2=GPS.lastNMEA();
  
  Serial.println(NMEA1);
  Serial.println(NMEA2);
  Serial.println("");
  
}
 
void clearGPS() {  //Clear old and corrupt data from serial port
  while(!GPS.newNMEAreceived()) { //Loop until you have a good NMEA sentence
    c=GPS.read();
  }
  GPS.parse(GPS.lastNMEA()); //Parse that last good NMEA sentence
  
  while(!GPS.newNMEAreceived()) { //Loop until you have a good NMEA sentence
    c=GPS.read();
  }
  GPS.parse(GPS.lastNMEA()); //Parse that last good NMEA sentence
   while(!GPS.newNMEAreceived()) { //Loop until you have a good NMEA sentence
    c=GPS.read();
  }
  GPS.parse(GPS.lastNMEA()); //Parse that last good NMEA sentence
  
}

I must have something wrong.

  while(!GPS.newNMEAreceived()) { //Loop until you have a good NMEA sentence
    c=GPS.read();
  }

This looks like it says

As long as there is no character received, read the device and ignore what you get.

Whereas this

  while(!GPS.newNMEAreceived());  // Loop until there is data to read

would dispense with reading what isn't there.

But I can't have that right.

But I can't have that right.

Correct. You don't.

As long as there is no character received, read the device and ignore what you get.

The function tests if the GPS instance has collected a complete sentence, not whether the serial port has received a character.

You are correct that the function will never return true, though, as the data that is read is discarded instead of being passed to the GPS instance.

Whereas this ... would dispense with reading what isn't there.

No. That (dumb) infinite loop will simply lock up the Arduino waiting for the instance to get a complete sentence while never passing the instance any data to (possibly) make the sentence complete.

You are correct, though, in that the code IS rubbish.

That's what I get for guessing!

Thanks, PaulS!

Got a bit longer on the code and tested it.
The GPS data is logging every 3 seconds.
I added a hardware function to the GPS unit.
There is 2 files made, and every time the Arduino starts up or reset the Arduino makes new files and not delete them
File nr1 - logfile, this file store the GGA and RMC raw data.
File nr2 - logfileGPS, this file stores spesial files in a format supported for Excel.

Power Voltage is the Input power to the Arduino.
Battery Voltage is the backup battery connected to the GPS unit.
The Power Voltage and Battery Voltage is only stored on the logfileGPS.

LINK TO FILE: http://kimlorentz.com/arduino/GPSlogger.txt

The files takes:
8% of program storage space
23% of dynamic storage space
On the Arduino Mega2560.