TinyGPS++ functionality question with NMEA sentences

I finally got my GPS setup and talking to my Arduino the way that I wanted to and I had a couple questions regarding the TinyGPS++ library.

My goal is to send the raw NMEA sentences over to the SoftwareSerial port and then pass that data along through my bluetooth module which is connected to the hardware serial port. I also need to extract the raw UTC GPS time from that NMEA sentence at the same time as well in order to do some other functions.

I've been able to extract the GPS time from the NMEA sentence with the TinyGPS++ library, but I haven't been able to figure out how to extract the GPS time and send the raw data to the HW serial port simultaneously.

Does anyone have suggestions on how I can do this?

I am currently passing the NMEA sentences along with this bit of code:

while (myGPS.available()){
    Serial.write(myGPS.read());
}

Add another line to that loop that feeds the same character to TinyGPS++, and tests for sentence completion in the usual way.

Post the code you are using with TinyGPS, using code tags.

When I tried to send the same character to the TinyGPS++, I got a bunch of garbage characters instead.

This is the full code that I'm using right now:

#include <TinyGPS++.h>

#include <SoftwareSerial.h>

SoftwareSerial myGPS(10,11);

TinyGPSPlus gps;

void setup() {
  Serial.begin(115200);
  myGPS.begin(38400);
}

void loop() {

  while (myGPS.available()){
    Serial.write(myGPS.read());
    gps.encode(myGPS.read());
  }
}

I also tried this, but then I was dropping data:

#include <TinyGPS++.h>

#include <SoftwareSerial.h>

char nextData;

SoftwareSerial myGPS(10,11);

TinyGPSPlus gps;

void setup() {
  Serial.begin(115200);
  myGPS.begin(38400);
}

void loop() {

  while (myGPS.available()){
    nextData=myGPS.read();
    Serial.write(nextData);
    gps.encode(nextData);
  }
}

SoftwareSerial used to have the exalted status of "worst and most unreliable Arduino software package available". Not sure what state it is in now, but I would not trust it at 38400 Baud.

I suspect that is your problem. There are replacements for SoftwareSerial (NewSoftSerial, NeoSWSerial) that work better. A forum member has also written a package called NeoGPS that may do everything you want.

My understanding is that SoftwareSerial was updated with NewSoftSerial since Arduino 1.0. I'll check out NeoGPS as well to see if that will do what I want as well

Take a look at this recent post from someone who is going through the same sorts of experiences.

fmemon:
My understanding is that SoftwareSerial was updated with NewSoftSerial since Arduino 1.0.

Yes, but it's still very inefficient, using about 95% of the CPU time whenever a character is received or sent (it can't do both at the same time).

If you do not plan on receiving anything fromSerial, you could connect the GPS TX pin to the Arduino RX pin 0 (Serial receive). You would have to disconnect it to upload new sketch, but it would be rock-solid reliable.

AltSoftSerial is next best, followed by NeoSWSerial. Read more here.

I'll check out NeoGPS as well to see if that will do what I want as well

Let me know if you have any questions. :slight_smile: It's smaller, faster, more reliable, more accurate, blah, blah, blah. Since you are looking at forwarding the characters, the NeoGPS example NMEAblink.ino is a good place to start. It shows the character-oriented technique instead of the typical "fix-oriented" technique.

I would like to ask about sending the raw NMEA data... Does another system really need that format, or are you in charge of the format of data it receives?

I ask, because sending the parsed information is 10x-80x fewer bytes. If this is a radio connection, sending the raw data is fairly wasteful. There's lots of duplication in the NMEA sentences. Sending text fields, like a CSV line is pretty good. Sending the binary fields is even better, but a little harder to debug. The code is simpler, though.

Cheers,
/dev

Just for fun, here's a NeoGPS+NeoSWSerial version of your sketch:

#include <NMEAGPS.h>
#include <NeoSWSerial.h>

NMEAGPS gps;
gps_fix fix;

NeoSWSerial gpsPort( 10, 11 );

void setup()
{
  Serial.begin( 9600 );
  gpsPort.begin( 9600 );
}

void loop()
{
  if (gpsPort.available()) {
    char c = gpsPort.read();
    Serial.write( c ); // forward it...

    // ... and parse it
    if (gps.decode( c ) == NMEAGPS::DECODE_COMPLETED) {
      fix = gps.fix(); // grab a copy of all the parsed pieces

      if (fix.valid.time) {
        Serial.println( fix.dateTime.seconds );
      } else {
        Serial.print( '.' );
      }
    }
  }
}

Your original sketch uses 6310 bytes of program space and 518 bytes of RAM.
The NeoGPS version uses 4738 bytes of program space and 314 bytes of RAM.

If you want to try it, NeoGPS and NeoSWSerial are available from the Arduino IDE Library Manager, under the menu Sketch -> Include Library -> Manage Libraries.

You might be interested in the example NMEAtimezone.ino. It shows how to use the date/time pieces in a fix, and how to offset UTC into a timezone, plus DST effects.

-dev:
If you do not plan on receiving anything fromSerial, you could connect the GPS TX pin to the Arduino RX pin 0 (Serial receive). You would have to disconnect it to upload new sketch, but it would be rock-solid reliable.

That ended up actually working for me! I connected the GPS TX to the Arduino RX and I was able to get the data that I expected on the serial monitor. I ended up passing that data over bluetooth to my phone using the TX line.

I would like to ask about sending the raw NMEA data... Does another system really need that format, or are you in charge of the format of data it receives?

I ask, because sending the parsed information is 10x-80x fewer bytes. If this is a radio connection, sending the raw data is fairly wasteful. There's lots of duplication in the NMEA sentences. Sending text fields, like a CSV line is pretty good. Sending the binary fields is even better, but a little harder to debug. The code is simpler, though.

The end goal is to pass the data from the Arduino to an app on my phone which will calculate/record lap times, speed, etc.

I'm also going to be recording other data as well at the same time. If I don't somehow grab the timestamp from the NMEA code, the GPS data, and the data from the sensors ends up drifting.

Here's the code of the data that I'll be passing along:

#include <SoftwareSerial.h>

#include <TinyGPS++.h>

TinyGPSPlus gps;

float tachFreq = 1333;
float thrtlVolt = 2.9;
float gearVolt = 3;
byte CRC = 0;
String dataStream;
char buff[100];
char inchar;
int count = 0;

void setup() {
  Serial.begin(115200);
}

void loop() {
  
  while (Serial.available()) {
    Serial.write(Serial.read());
  }

  dataStream = "$RC3,,";
  dataStream.concat(count);
  dataStream.concat(",,,,,,,");
  dataStream.concat(tachFreq);
  dataStream.concat(",,");
  dataStream.concat(thrtlVolt);
  dataStream.concat(",");
  dataStream.concat(gearVolt);
  dataStream.concat(",,,,,,,,,,,,,*");

  // calculate CRC here > add to dataStream

  Serial.println(dataStream);
  count = count + 1;

}

EDIT:
The GPS is setup to only send RMC+GGA right now at 10Hz. Baud rate was upped to 115200 to support the extra data throughput

fmemon:
I connected the GPS TX to the Arduino RX and I was able to get the data that I expected on the serial monitor.

Great!

fmemon:
The end goal is to pass the data from the Arduino to an app on my phone which will calculate/record lap times, speed, etc.

There are several NeoGPS lap timers here, with various degrees of complexity.

If I don't somehow grab the timestamp from the NMEA code, the GPS data, and the data from the sensors ends up drifting.

Yes, the Arduino clock frequency is not as good as the GPS atomic clock. It will drift. You can see in the sketch below how to get the UTC hours, minutes and seconds from the fix structure that NeoGPS fills out for you.

Here's the code of the data that I'll be passing along:

Great, that gives us a better idea of what you're trying to do.

Here's a NeoGPS version:

#include <NMEAGPS.h>

NMEAGPS gps;
gps_fix fix;  // the structure that holds all the parsed pieces

#define gpsPort Serial  // a simple substitution
//  Later, you can use a different serial object if you want.
//  Change the definition of 'gpsPort' and the rest of your code still works.
//  For example:
//    AltSoftSerial gpsPort; // GPS TX to UNO pin 9
//  Or
//    #define gpsPort Serial1  // on a Mega

float tachFreq  = 1333.0;  // remember that floats always have a decimal point
float thrtlVolt = 2.9;
float gearVolt  = 3.0;

int count = 0;

void setup() {
  Serial.begin( 115200 );
  //gpsPort.begin( 9600 );  // Uncomment this if you change gpsPort above.
}

void loop() {

  while (gps.available( gpsPort )) { // NeoGPS reads the port for you
    fix = gps.read();  // after enough reads, a fix structure is ready (10Hz)
    sendMessage();  // now that we have new GPS data, send a message
  }
}

#define CF(x) ((const char *)F(x)) // a helper "function" for below

void sendMessage()
{
  // Instead of making one big buffer, this deals with each piece.
  //    The `sendPiece` function below prints the piece *and* calculates
  //    a checksum for that piece.  The checksum accumulates all pieces' checksums.

  char    buf[16]; // for just one piece
  uint8_t CS = 0;

  strcpy_P( buf, CF("$RC3,,") );  // a string constant (from FLASH memory) is copied into the buffer
  sendPiece( buf, CS );           //   This saves RAM, too.

  itoa( count, buf, 10 );         // an integer is written to the buffer
  sendPiece( buf, CS );

  strcpy_P( buf, CF(",,,,,,,") );
  sendPiece( buf, CS );

  //--------------------------
  // This might be where you send pieces of the fix structure.  Speed?

  //  lat/lon
  if (fix.valid.location) {
    dtostrf( fix.latitude(), 1, 6, buf ); // a float is written to the buffer
    sendPiece( buf, CS );
  }
  sendChar( ',', CS );
  if (fix.valid.location) {
    dtostrf( fix.longitude(), 1, 6, buf );
    sendPiece( buf, CS );
  }
  sendChar( ',', CS );

  // UTC time
  if (fix.valid.time) {
    itoa( fix.dateTime.seconds, buf, 10 ); // an integer
    sendPiece( buf, CS );
 
    sendChar( '.', CS ); // and the hundredths of seconds
    if (fix.dateTime_cs < 10) // leading zero
      sendChar( '0', CS );
    itoa( fix.dateTime_cs, buf, 10 );
    sendPiece( buf, CS );
    
    // minutes & hours too?  Leading zeroes?  ":" separators?
  }
  sendChar( ',', CS );
  //--------------------------

  dtostrf( tachFreq, 1, 2, buf ); // a float
  sendPiece( buf, CS );

  strcpy_P( buf, CF(",,") );
  sendPiece( buf, CS );

  dtostrf( thrtlVolt, 1, 2, buf );
  sendPiece( buf, CS );

  sendChar( ',', CS );

  dtostrf( gearVolt, 1, 2, buf );
  sendPiece( buf, CS );

  strcpy_P( buf, CF(",,,,,,,,,,,,,*") ); // Are you sure you want to add '*' to the CS?
  sendPiece( buf, CS );

  // Now send the two hex digit CS
  if (CS < 0x10)
    Serial.print( '0' ); // leading zero...
  Serial.println( CS, HEX ); // ... because this does not print leading zeroes

  count = count + 1;
}


void sendPiece( char *buf, uint8_t & cs ) // '&' means it modifies the passed-in variable
{
  Serial.print( buf );  // send just this much...

  //  ... and calculate the checksum for this piece.
  for (uint8_t i = 0; buf[i] != 0; i++) {
    cs ^= buf[i]; // same as 'cs = cs xor buf[i];'.  Caret is XOR operator
  }
}

void sendChar( char c, uint8_t & cs ) // a helper function to send one char
{
  char buf[2];
  buf[ 0 ] = c;   // assign one character to the first element of the array...
  buf[ 1 ] = 0;   // ... and NUL-terminate the array (i.e. a C string).

  sendPiece( buf, cs );
}

Using String can be a recipe for disaster, so I also wanted to show you how to use C strings, aka char arrays. Eliminating String also saves 1600 bytes of program space. Official documentation here for dtostrf and itoa.

Notice how it prints each piece, instead of accumulating everything in one big buffer. This saves a lot of RAM. The checksum is also calculated for each piece, and accumulates as the pieces are sent. Now the buffer only has to be as big as the biggest piece, not the whole line.

The GPS is setup to only send RMC+GGA right now at 10Hz. Baud rate was upped to 115200 to support the extra data throughput

When you know what pieces you want to send, you can use this table to decide which NMEA sentences you really need. Maybe just RMC?

You can configure NeoGPS to parse only those sentences and fields. Everything else is quickly skipped.
This will eliminate more program space and RAM. It will also reduce the processing load on the Arduino, making your sketch run even faster.

Cheers,
/dev

Thanks for the code. I'm going line by line to understand how this thing works, since I've never done coding like what you're showing before (I'm still pretty new at all of this)

When you know what pieces you want to send, you can use this table to decide which NMEA sentences you really need. Maybe just RMC?

The app that's getting the data requires that the full RMC+GGA data come through, otherwise, it just assumes there's no satellite fix present. In my initial trial run, I setup the GPS to only send RMC sentences only, which the app wouldn't recognize, since the satellite info wasn't there. After adding in GGA, the app acknowledged the GPS data.

That's pretty much the only reason why I need to get that bit of data sent across as well. I'm trying to figure out if there is a way around this limitation at this time, since only having to send parsed data would work out much better for me, maybe even allowing me to increase the GPS refresh rate as well, if I wanted to do that.

strcpy_P( buf, CF(",,,,,,,,,,,,,*") ); // Are you sure you want to add '*' to the CS?

The asterisk and the $ sign actually don't get sent over into the CS calculation. In my code, I have a search function which starts at the $ all the way up to the asterisk, but doesn't include them for the CS. This is how I was calculating the CS:

byte getCRC(String temp) {
  byte start_with = 0;
  byte end_with = 0;
  byte index = 0;

  CRC = 0;

  temp.toCharArray(buff, 100);

  for ( index = 0; index < 100; index++) {
    inchar = buff[index];

    if ( inchar == '

) {
     start_with = index;
   }

if (inchar == '*') {
     end_with = index;
   }

}

for (byte x = start_with + 1; x < end_with; x++) { // XOR every character in between '


 and '*'
    CRC = CRC ^ buff[x];
  }

  return CRC;
}

The end goal is to pass the data from the Arduino to an app on my phone which will calculate/record lap times, speed, etc.

I assumed that you wrote that app. Did you? If not, which app is it?

If it's RaceChrono, it has several formats. I saw documentation for the $RC3 message you are sending. Are you adding the IMU data to the message?

The asterisk and the $ sign actually don't get sent over into the CS calculation. In my code

Good, that's a simple change. Just move the asterisk out of the strcpy and print it separately.

  strcpy_P( buf, CF(",,,,,,,,,,,,,") ); // No '*'
  sendPiece( buf, CS );
  Serial.print( '*' ); // send it now, without using it in the CS.

-dev:
If it's RaceChrono, it has several formats. I saw documentation for the $RC3 message you are sending. Are you adding the IMU data to the message?

That's exactly the app I was planning on using. I'll eventually be adding an IMU (looking at the MPU6050 accelerometer/gyroscope module), but right now, I'm trying to figure out this little bit before I move forward.

From what I could see, the $RC3 message doesn't take the lat/long coordinates. Is there another way that you know would work with that app?

After a closer look, it seems like it may allow some binary protocols from the "approved" external GPS devices. As you have discovered, it will function with two required NMEA sentences, GGA and RMC.

Based on this quick look, I would suggest just forwarding the 10Hz NMEA data. As it passes through, you can extract the UTC timestamp with NeoGPS and put it into your $RC3 sentences. There are NeoGPS functions that return fractional UTC seconds for you to use when the $RC3 sentence rate is higher than the GPS rate.

Here's a modified version of loop to both forward and parse NMEA data:

void loop() {

  if (gpsPort.available()) {
    char c = gpsPort.read();

    Serial.write ( c );    // forward it
    gps   .handle( c );    // parse it

    // Did 'handle' finish making a new fix structure?
    if (gps.available() && (c == 0x0A)) { // wait for last LF char
      fix = gps.read();    // yes, a fix structure is ready (10Hz)
      sendMessage();       // now that we have new GPS data, send a message
    }
  }
}

When you're ready, I can talk about configuring NeoGPS to only parse the time and how you use the fix.dateTime to make a timestamp for the $RC3 sentence.

I ended up going through your code and I think I somewhat figured out how to get the data that I need across.

I added the specific code used to measure the frequency of the tachometer line, and the analog reads for the throttle and gear positions.

It looks like everything is working how I would expect it to work with the time stamps of the GPS matching the time stamps of the data logger, so there's no drift in the overall setup.

Here's the modified code now:

#include <FreqMeasure.h>

#include <NMEAGPS.h>

NMEAGPS gps;
gps_fix fix;  // the structure that holds all the parsed pieces

#define gpsPort Serial  // a simple substitution

/* ~~~~ Pin configurations ~~~~ */
const byte aThrottleVoltPin = A0;
const byte aGearVoltPin = A1;

double tach = 0;
float tachFreq = 0.0;
int thrtlVolt = 0;
int gearVolt  = 0;

void setup() {
  pinMode(aThrottleVoltPin, INPUT);
  pinMode(aGearVoltPin, INPUT);

  Serial.begin(115200);
  FreqMeasure.begin();

  Serial.begin( 115200 );
  FreqMeasure.begin();
}

void loop() {

  if (gpsPort.available()) {
    char c = gpsPort.read();

    Serial.write ( c );    // forward it
    gps   .handle( c );    // parse it

    // Did 'handle' finish making a new fix structure?
    if (gps.available() && (c == 0x0A)) { // wait for last LF char
      fix = gps.read();    // yes, a fix structure is ready (10Hz)

      thrtlVolt = analogRead(aThrottleVoltPin);
      gearVolt = analogRead(aGearVoltPin);

      if (FreqMeasure.available()) {
        tach = FreqMeasure.read();
        tachFreq = FreqMeasure.countToFrequency(tach);
      }

      sendMessage();       // now that we have new GPS data, send a message
    }
  }
}

#define CF(x) ((const char *)F(x)) // a helper "function" for below

void sendMessage()
{
  // Instead of making one big buffer, this deals with each piece.
  //    The `sendPiece` function below prints the piece *and* calculates
  //    a checksum for that piece.  The checksum accumulates all pieces' checksums.

  char    buf[16]; // for just one piece
  uint8_t CS = 0;

  Serial.print("$");

  strcpy_P( buf, CF("RC3,") );  // a string constant (from FLASH memory) is copied into the buffer
  sendPiece( buf, CS );           //   This saves RAM, too.

  // UTC time
  if (fix.valid.time) {
    itoa( fix.dateTime.hours, buf, 10 ); // hours
    sendPiece( buf, CS );

    if (fix.dateTime.minutes < 10) // leading zero
      sendChar( '0', CS );
    itoa( fix.dateTime.minutes, buf, 10 ); // minutes
    sendPiece( buf, CS );

    itoa( fix.dateTime.seconds, buf, 10 ); // seconds
    sendPiece( buf, CS );

    sendChar( '.', CS ); // and the hundredths of seconds
    if (fix.dateTime_cs < 10) // leading zero
      sendChar( '0', CS );
    itoa( fix.dateTime_cs, buf, 10 );
    sendPiece( buf, CS );
  }

  strcpy_P( buf, CF(",,,,,,,,") );
  sendPiece( buf, CS );

  dtostrf( tachFreq, 1, 2, buf ); // a float
  sendPiece( buf, CS );

  strcpy_P( buf, CF(",,") );
  sendPiece( buf, CS );

  itoa( thrtlVolt, buf, 10 );
  sendPiece( buf, CS );

  sendChar( ',', CS );

  itoa( gearVolt, buf, 10 );
  sendPiece( buf, CS );

  strcpy_P( buf, CF(",,,,,,,,,,,,,") );
  sendPiece( buf, CS );

  Serial.print("*");

  // Now send the two hex digit CS
  if (CS < 0x10)
    Serial.print( '0' ); // leading zero...
  Serial.println( CS, HEX ); // ... because this does not print leading zeroes

}


void sendPiece( char *buf, uint8_t & cs ) // '&' means it modifies the passed-in variable
{
  Serial.print( buf );  // send just this much...

  //  ... and calculate the checksum for this piece.
  for (uint8_t i = 0; buf[i] != 0; i++) {
    cs ^= buf[i]; // same as 'cs = cs xor buf[i];'.  Caret is XOR operator
  }
}

void sendChar( char c, uint8_t & cs ) // a helper function to send one char
{
  char buf[2];
  buf[ 0 ] = c;   // assign one character to the first element of the array...
  buf[ 1 ] = 0;   // ... and NUL-terminate the array (i.e. a C string).

  sendPiece( buf, cs );
}

Let me know what you think about it.

Looks good! One minor note... This

    Serial.print("*");

...should be this:

    Serial.print( '*' );

Single-character literals are delimited by the apostrophe. Multiple-character literals are delimited by double-quotes, and usually wrapped with the F or CF macros to save RAM. A double-quoted literal of one character actually uses two bytes: one for the character, and one for the NUL terminator character (a zero).

The bottom line is that the single-quoted character uses no RAM, and the double-quoted character uses 2 bytes of RAM, and is a little slower to print. Not a big deal unless you do this a lot or are really short on RAM.

I changed that part of the code up as well.

I just got my accelerometer which I'm going to start messing around with. I'm sure I'll have some questions regarding that as well. I'll start up a new thread if I have any specific questions with the accelerometer.

Thanks again for all the help you gave me with the code.