GPS parsing problem

G'day all,

I'm sure I don't need to tell you I'm a "Newb" because I'm here asking a question!

I've been trying to parse GPS data from a 3G phone module but only when called. The output from the module is formatted as GPSACP and not the generic GPRMC sentence as would be the case with dedicated free running GPS. I have no trouble getting the data from the module as required but without re-inventing the wheel and writing the parsing code myself I can't figure out how to use for example Tiny GPS to simplify the task. Is there a way to get this Library to recognise the GPSACP: sentence from the 3G module? The code below is based on the work of others so I'm confident that with a valid GPRMC sentence output it would work fine.

Any assitance would be greatly appreciated.

Cheers
Greg.

// In order for this sketch to work, you will need to download
//TinyGPS libraries from arduiniana.org and put them
// into the hardware->libraries folder in your ardiuno directory.
// Here are the lines of code that point to those libraries.

#include <TinyGPS.h>

TinyGPS gps;

//void getgps(TinyGPS &gps);

void setup()
{
// This is the serial rate for your terminal program. It must be this
// fast because we need to print everything before a new sentence
// comes in. If you slow it down, the messages might not be valid and
// you will likely get checksum errors.
Serial.begin(115200);

Serial.println("");
Serial.println("GPS Test Example Sketch");
Serial.println("");
Serial.println("ATE0"); //turn echo off
delay(5000); //wait for 3G module
Serial.println("AT$GPSP=1");
delay(5000); //wait for 3G module
}

// This is the main loop of the code. All it does is check for data on
// the RX pin of the ardiuno, makes sure the data is valid NMEA sentences,
// then jumps to the getgps() function.
void loop()
{
Serial.println ("AT$GPSACP"); // Call for GPS data
while(Serial.available()) // While there is data on the RX pin...
{
int c = Serial.read(); // load the data into a variable...
if(gps.encode(c)) // if there is a new valid sentence...
{
getgps(gps); // then grab the data.
}
}

delay(10000); //delay before next loop
}

// The getgps function will get and print the values we want.

void getgps(TinyGPS &gps)

{
// To get all of the data into varialbes that you can use in your code,
// all you need to do is define variables and query the object for the
// data. To see the complete list of functions see keywords.txt file in
// the TinyGPS and NewSoftSerial libs.

// Define the variables that will be used
float latitude, longitude;
// Then call this function
gps.f_get_position(&latitude, &longitude);
// You can now print variables latitude and longitude
Serial.print("Lat/Long: ");
Serial.print(latitude,5);
Serial.print(", ");
Serial.println(longitude,5);

// Same goes for date and time
/*int year;
byte month, day, hour, minute, second, hundredths;
gps.crack_datetime(&year,&month,&day,&hour,&minute,&second,&hundredths);
// Print data and time
Serial.print("Date: "); Serial.print(month, DEC); Serial.print("/");
Serial.print(day, DEC); Serial.print("/"); Serial.print(year);
Serial.print(" Time: "); Serial.print(hour, DEC); Serial.print(":");
Serial.print(minute, DEC); Serial.print(":"); Serial.print(second, DEC);
Serial.print("."); Serial.println(hundredths, DEC);
//Since month, day, hour, minute, second, and hundr
*/
// Here you can print the altitude and course values directly since
// there is only one value for the function
//Serial.print("Altitude (meters): "); Serial.println(gps.f_altitude());
// Same goes for course
Serial.print("Course (degrees): "); Serial.println(gps.f_course());
// And same goes for speed
Serial.print("Speed(knots): "); Serial.println(gps.f_speed_knots());
Serial.println();

// Here you can print statistics on the sentences.
//unsigned long chars;
//unsigned short sentences, failed_checksum;
//gps.stats(&chars, &sentences, &failed_checksum);
//Serial.print("Failed Checksums: ");Serial.print(failed_checksum);
//Serial.println(); Serial.println();
}

Do you have a sample string sent from the device?

I have no trouble getting the data from the module as required but without re-inventing the wheel and writing the parsing code myself I can't figure out how to use for example Tiny GPS to simplify the task.

You can't. TinyGPS doesn't know about GPSACP mode.

Is there a way to get this Library to recognise the GPSACP: sentence

Yes. You can add code to TinyGPS to make it parse GPSACP sentences, IF you know what all the values mean.

If it is a text string in the same manner as the other NMEA GPS sentences, then there isn't much
difficulty in adding it as a type of sentence to be processed by the parser.

You basically count the commas, and identify what is between each pair of commas as an integer or
floating point number or character code or whatever it is.

On the other hand, it might be something completely different. I don't know.

Gents,

Below is the response from the GPS Module. The string contents are as follows:-

UTC Time , Latitude, Longitude, HDOP, Altitude, Fix, COG, Speed(Km), Speed (Knots), Date, NSAT

I'm interested to know if the changes to Tiny GPS would need to be made in the .h file, or the .cpp file... Or both? (Never edited a library before)... Also, I would expect that simply adding the string definition to those that already exist in the Library would be the best option.

Again, any help greatly appreciated.

Cheers
Greg.

$GPSACP:012647.000,2728.3033S,15302.3369E,1.50,-52.0,3,156.66,0.00,0.00,240113,04

I'm interested to know if the changes to Tiny GPS would need to be made in the .h file, or the .cpp file... Or both?

The source (cpp) file.

Look at the source code. It should be fairly simple (and obvious) how to parse a different kind of record.

In what form do you want the data and what do you plan to do with it? As a simple parsing start, load the below code into the arduino, copy the sample string you posted, paste the string in the serial monitor (ctrl-v), then send it to the arduino.

//zoomkat 3-5-12 simple delimited ',' string parce 
//from serial port input (via serial monitor)
//and print result out serial port
// CR/LF could also be a delimiter

String readString;

void setup() {
  Serial.begin(9600);
  Serial.println("serial delimit test 1.0"); // so I can keep track of what is loaded
}

void loop() {

  //expect a string like wer,qwe rty,123 456,hyre kjhg,
  //or like hello world,who are you?,bye!,
  
  if (Serial.available())  {
    char c = Serial.read();  //gets one byte from serial buffer
    if (c == ',') {
      //do stuff
      Serial.println(readString); //prints string to serial port out
      readString=""; //clears variable for new input      
     }  
    else {     
      readString += c; //makes the string readString
    }
  }
}

This PDF describes the format of the GPSACP sentence:
http://www.gatetel.com/PDF/GPSgprs/GPRS_Protocol_description.pdf

Pete

Hi all,

Sorry for the delay in getting back to this... after studying all your input I've been trying different solutions and have come up with one that seems quite elegant at first glance... Trouble is once incorporated into my complete code it doesn't work. Everything appears fine but in the "real world" situation the variables areeither not being populated, or the results are not being printed to the serial bus. For bothe the simulator and serial monitor run I manually enter the GPS response ($GPSACP: 234423.31,2728.3913S,15302.3418E,1.00,38.81,3,0,00,0.01,0.02,180313,07) into the serial monitor input and the correctly formatted output appears on the output.

The Parsing/printing code which works both on an arduino using the serial monitor, and in the simulator is here.

#include <avr/pgmspace.h>
  #define maxLength 80                                 // Set incoming string max length

  
 
  String readString;
  String Lat;
  String Lon;
  String COG;
  String Speed;
  int ComCount = 0;
  
void setup() {
  
  Serial.begin(9600);
  delay (5000);
  Serial.println("GPS Test");
  Serial.println ("AT$GPSACP");
  delay(1000);
}

void loop()  { 

   
  if (Serial.available())  {
    char c = Serial.read();  //gets one byte from serial buffer
    if (c ==':')  {
      readString="";
    }
    
    //delay(1000);
    
    if (c == ',') {
      ComCount++;

      if (ComCount == 1)  {
        readString=""; //clears variable for new input      
      }
    
    //delay(1000);
    
     

      if (ComCount == 2)  {
        Lat = readString;
        
        readString=""; //clears variable for new input 
      }
    
    //delay(1000);
    
      

      if (ComCount == 3)  {
        Lon = readString;
        
        readString=""; //clears variable for new input      
      }
    
    //delay(1000);
    
    

      if (ComCount == 4)  {
        readString=""; //clears variable for new input      
      }
    
    //delay(1000);
    
    

      if (ComCount == 5)  {
        readString=""; //clears variable for new input      
      }
    
    //delay(1000);
    
    

      if (ComCount == 6)  {
        /*if (c += '1')  {
         Fix = 0;
         }
         else Fix = 1;
         */       
        
      readString=""; //clears variable for new input
      }
    
    //delay(1000);
    
    

      if (ComCount == 7)  {
       COG = readString;
        
        readString=""; //clears variable for new input      
      }
    
    //delay(1000);
    
     

      if (ComCount == 8)  {
        readString=""; //clears variable for new input      
      }
    
    //delay(1000);
    
    

      if (ComCount == 9)  {
        Speed = readString;
        
        readString=""; //clears variable for new input      
      }
    
    //delay(1000);
    
     

      if (ComCount == 10)  {
        readString=""; //clears variable for new input
       Serial.println ("Lat " + Lat);
   Serial.println ("Lon " + Lon);
   Serial.println ("Speed " + Speed + " Knots"); 
   Serial.println ("COG " + COG); 
        ComCount = 0;     
      }
    
   
   

    
    
    }
    else {     
      readString += c; //makes the string readString
    }  
  }

}

The subroutines I'm actually calling are here... (first section calls the GPS subroutine then should print the data stored in the appropriate variables???)

if (inString.indexOf("Locate") >=0) {                        // look for a command in the string       

      inString = "";                                         // Clear the string
      GPS();

      Serial.println ("AT+CMGS=0430340511");             //Send SMS to this number
  
      showString(PSTR("Current Location, Course, & Speed\r\n"));
      Serial.println("");
      Serial.println ("Lat " + Lat);
      Serial.println ("Lon " + Lon);
      Serial.println ("COG " + COG);
      Serial.println ("Speed " + Speed + " Knots");
      Serial.println ("\x1A");                           //Delay to accomodate message send.
      delay(2000);                                       //Delay to accomodate message send.

      inString = "";                                               // Clear the string

    }

void GPS()  { 

  delay(1000);
  Serial.println ("AT$GPSACP");
  delay(2000);
  
  if (Serial.available())  {
    char c = Serial.read();  //gets one byte from serial buffer
    if (c ==':')  {
      readString="";
      ComCount = 0;
    }
    if (c == ',') {
      ComCount++;

      if (ComCount == 1)  {
        readString=""; //clears variable for new input      
      } 

      if (ComCount == 2)  {
        Lat = readString;
        readString=""; //clears variable for new input 
      }  

      if (ComCount == 3)  {
        Lon = readString;
        readString=""; //clears variable for new input      
      }

      if (ComCount == 4)  {
        readString=""; //clears variable for new input      
      }

      if (ComCount == 5)  {
        readString=""; //clears variable for new input      
      }

      if (ComCount == 6)  {      
        readString=""; //clears variable for new input
      }

      if (ComCount == 7)  {
        COG = readString;
        readString=""; //clears variable for new input      
      } 

      if (ComCount == 8)  {
        readString=""; //clears variable for new input      
      }

      if (ComCount == 9)  {
        Speed = readString;
        readString=""; //clears variable for new input      
      } 

      if (ComCount == 10)  {
        readString=""; //clears variable for new input 
        ComCount = 0;     
      }
    }
    else {     
      readString += c; //makes the string readString
      
      delay(1000);
    }  
  }
}

The delays are simply for stability. The GPS/3G module responses are fairly quick but I want reliability over absolute speed.

It seems to me that from my testing with the first block of code I've included that perhaps the issue is with the response from the GPS not actually being available on the serial bus??? Other comands that come in via text message are recognised, and in fact the entire sequence is called in this manner. I can also confirm that watching the serial data output from both the arduino and the GPS/3G module the correct calls and responses are present.

Cheers
Greg.

#7 below might help alot.

http://arduino.cc/forum/index.php/topic,148850.0.html

Thanks for the tip... modified post as suggested.

Cheers
Greg

There is the TinyGPS library that handles parsing GPS data without using crutches (otherwise known as the String class). Why not use it?

Or, at least refer to it to see how parsing is done.

Thanks PaulS,

This was discussed at the beginning of the topic. TinyGPS doesn't recognise or handle GPSACP strings natively and I figured it was just as easy (and far more educational) to attack the problem head on.

Cheers
Greg.

I figured it was just as easy (and far more educational) to attack the problem head on.

But, at least the TinyGPS library provides a skeleton. Why not add the ability to parse GPSACP sentences to that library?

All those Strings coming and going are trashing the limited memory that you have.

Thanks PaulS,

So you think the issue could be related to memory?... OK, I'll relook at TinyGPS and see if I can figure out how to add a sentence type to it...

Cheers
Greg

O.K...., Still pretty new to all this...

Below is my modified version of Tiny GPS... Not sure if it's going to work as I've never really dealt with libraries before but I'm happy to give it a go! Any suggestions would be greatly appreciated...

Cheers
Greg.

/*
  TinyGPS - a small GPS library for Arduino providing basic NMEA parsing
  Copyright (C) 2008-9 Mikal Hart
  All rights reserved.

  Satellite Count Mod - by Brett Hagman
  http://www.roguerobotics.com/

  This library is free software; you can redistribute it and/or
  modify it under the terms of the GNU Lesser General Public
  License as published by the Free Software Foundation; either
  version 2.1 of the License, or (at your option) any later version.

  This library is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
  Lesser General Public License for more details.

  You should have received a copy of the GNU Lesser General Public
  License along with this library; if not, write to the Free Software
  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
*/

#include "WProgram.h"
#include "TinyGPS.h"

#define _GPRMC_TERM   "GPRMC"
#define _GPGGA_TERM   "GPGGA"
#define _GPGSV_TERM   "GPGSV"
#define _GPGSA_TERM   "GPGSA"
#define _GPSACP_TERM   "GPSACP"


TinyGPS::TinyGPS()
:  _time(GPS_INVALID_TIME)
,  _date(GPS_INVALID_DATE)
,  _latitude(GPS_INVALID_ANGLE)
,  _longitude(GPS_INVALID_ANGLE)
,  _altitude(GPS_INVALID_ALTITUDE)
,  _speed(GPS_INVALID_SPEED)
,  _course(GPS_INVALID_ANGLE)
,  _satsinview(0)
,  _satsused(0)
,  _fixtype(GPS_FIX_NO_FIX)
,  _last_time_fix(GPS_INVALID_FIX_TIME)
,  _last_position_fix(GPS_INVALID_FIX_TIME)
,  _parity(0)
,  _is_checksum_term(false)
,  _sentence_type(_GPS_SENTENCE_OTHER)
,  _term_number(0)
,  _term_offset(0)
#ifndef _GPS_NO_STATS
,  _encoded_characters(0)
,  _good_sentences(0)
,  _failed_checksum(0)
#endif
{
  _term[0] = '\0';
}

//
// public methods
//

bool TinyGPS::encode(char c)
{
  bool valid_sentence = false;

  ++_encoded_characters;
  switch(c)
  {
  case ',': // term terminators
    _parity ^= c;
  case '\r':
  case '\n':
  case '*':
    if (_term_offset < sizeof(_term))
    {
      _term[_term_offset] = 0;
      valid_sentence = term_complete();
    }
    ++_term_number;
    _term_offset = 0;
    _is_checksum_term = c == '*';
    return valid_sentence;

  case '

: // sentence begin
    _term_number = _term_offset = 0;
    _parity = 0;
    _sentence_type = _GPS_SENTENCE_OTHER;
    _is_checksum_term = false;
    return valid_sentence;
  }

// ordinary characters
  if (_term_offset < sizeof(_term) - 1)
    _term[_term_offset++] = c;
  if (!_is_checksum_term)
    _parity ^= c;

return valid_sentence;
}

#ifndef _GPS_NO_STATS
void TinyGPS::stats(unsigned long *chars, unsigned short *sentences, unsigned short *failed_cs)
{
  if (chars) *chars = _encoded_characters;
  if (sentences) *sentences = _good_sentences;
  if (failed_cs) *failed_cs = _failed_checksum;
}
#endif

//
// internal utilities
//
int TinyGPS::from_hex(char a)
{
  if (a >= 'A' && a <= 'F')
    return a - 'A' + 10;
  else if (a >= 'a' && a <= 'f')
    return a - 'a' + 10;
  else
    return a - '0';
}

unsigned long TinyGPS::parse_decimal()
{
  char *p = _term;
  bool isneg = *p == '-';
  if (isneg) ++p;
  unsigned long ret = 100UL * gpsatol(p);
  while (gpsisdigit(*p)) ++p;
  if (*p == '.')
  {
    if (gpsisdigit(p[1]))
    {
      ret += 10 * (p[1] - '0');
      if (gpsisdigit(p[2]))
        ret += p[2] - '0';
    }
  }
  return isneg ? -ret : ret;
}

unsigned long TinyGPS::parse_degrees()
{
  char *p;
  unsigned long left = gpsatol(_term);
  unsigned long tenk_minutes = (left % 100UL) * 10000UL;
  for (p=_term; gpsisdigit(*p); ++p);
  if (p == '.')
  {
    unsigned long mult = 1000;
    while (gpsisdigit(
++p))
    {
      tenk_minutes += mult * (*p - '0');
      mult /= 10;
    }
  }
  return (left / 100) * 100000 + tenk_minutes / 6;
}

// Processes a just-completed term
// Returns true if new sentence has just passed checksum test and is validated
bool TinyGPS::term_complete()
{
  if (_is_checksum_term)
  {
    byte checksum = 16 * from_hex(_term[0]) + from_hex(_term[1]);
    if (checksum == _parity)
    {
#ifndef _GPS_NO_STATS
      ++_good_sentences;
#endif
      _last_time_fix = _new_time_fix;
      _last_position_fix = _new_position_fix;

switch(_sentence_type)
      {
        case _GPS_SENTENCE_GPRMC:
          _time      = _new_time;
          _date      = _new_date;
          _latitude  = _new_latitude;
          _longitude = _new_longitude;
          _speed    = _new_speed;
          _course    = _new_course;
          break;
        case _GPS_SENTENCE_GPGGA:
          _altitude  = _new_altitude;
          _time      = _new_time;
          _latitude  = _new_latitude;
          _longitude = _new_longitude;
          break;
        case _GPS_SENTENCE_GPGSV:
          _satsinview = _new_satsinview;
          break;
        case _GPS_SENTENCE_GPGSA:
          _satsused = _new_satsused;
          _fixtype = _new_fixtype;
          break;
  case _GPS_SENTENCE_GPSACP:
          _time      = _new_time;
          _date      = _new_date;
          _latitude  = _new_latitude;
          _longitude = _new_longitude;
          _speed    = _new_speed;
          _course    = _new_course;
          break;

}
      return true;
    }

#ifndef _GPS_NO_STATS
    else
      ++_failed_checksum;
#endif
    return false;
  }

// the first term determines the sentence type
  if (_term_number == 0)
  {
    if (!gpsstrcmp(_term, _GPRMC_TERM))
      _sentence_type = _GPS_SENTENCE_GPRMC;
    else if (!gpsstrcmp(_term, _GPGGA_TERM))
      _sentence_type = _GPS_SENTENCE_GPGGA;
    else if (!gpsstrcmp(_term, _GPGSV_TERM))
    {
      _sentence_type = _GPS_SENTENCE_GPGSV;
    }
    else if (!gpsstrcmp(_term, _GPGSA_TERM))
    {
      _sentence_type = _GPS_SENTENCE_GPGSA;
      _new_satsused = 0;
    }

else if (!gpsstrcmp(_term, _GPSACP_TERM))

{
      _sentence_type = _GPS_SENTENCE_GPSACP;
    }

else
      _sentence_type = _GPS_SENTENCE_OTHER;
    return false;
  }

if (_sentence_type == _GPS_SENTENCE_GPGSV)
  {
    if (_term_number == 3 && _term[0])
    {
      // we've got our number of sats
      // NOTE: we will more than likely hit this a few times in a row, because
      // there are usually multiple GPGSV sentences to describe all of the sats
      _new_satsinview = (unsigned char) gpsatol(_term);
    }
  }
  else if (_sentence_type == _GPS_SENTENCE_GPGSA)
  {
    if (_term_number == 2 && _term[0])  // Fix type
    {
      _new_fixtype = (unsigned char) gpsatol(_term);
    }
    else if (_term_number >= 3 && _term_number <= 14 && _term[0]) // Count our sats used
    {
      _new_satsused++;
    }
//    if (_term_number == 15)  // PDOP
//    if (_term_number == 16)  // HDOP
//    if (_term_number == 17)  // VDOP
  } 
  else if (_sentence_type != _GPS_SENTENCE_OTHER && _term[0])
  {
    switch((_sentence_type == _GPS_SENTENCE_GPGGA ? 200 : 100) + _term_number)
    {
      case 101: // Time in both sentences
      case 201:
        _new_time = parse_decimal();
        _new_time_fix = millis();
        break;
      case 103: // Latitude
      case 202:
        _new_latitude = parse_degrees();
        _new_position_fix = millis();
        break;
      case 104: // N/S
      case 203:
        if (_term[0] == 'S')
          _new_latitude = -_new_latitude;
        break;
      case 105: // Longitude
      case 204:
        _new_longitude = parse_degrees();
        break;
      case 106: // E/W
      case 205:
        if (_term[0] == 'W')
          _new_longitude = -_new_longitude;
        break;
      case 107: // Speed (GPRMC)
        _new_speed = parse_decimal();
        break;
      case 108: // Course (GPRMC)
        _new_course = parse_decimal();
        break;
      case 109: // Date (GPRMC)
        _new_date = gpsatol(_term);
        break;
      case 209: // Altitude (GPGGA)
        _new_altitude = parse_decimal();
        break;
    }
  }

return false;
}

long TinyGPS::gpsatol(const char *str)
{
  long ret = 0;
  while (gpsisdigit(*str))
    ret = 10 * ret + *str++ - '0';
  return ret;
}

int TinyGPS::gpsstrcmp(const char *str1, const char *str2)
{
  while (*str1 && *str1 == *str2)
    ++str1, ++str2;
  return *str1;
}

You need to be aware that GPS doesn't work indoors, and most GPS devices will output NMEA strings with 10 or 15 commas
and no values between them.

You need to firstly capture the actual output text from the GPS device and echo it to your computer serial monitor to
see what it looks like, and make sure it is the same as what those GPSAPC strings you are expecting, look like.

You need to be aware that GPS doesn't work indoors

Depends which kind of building you are in and which GPS you are using. My Adafruit Ultimate GPS and Garmin GPSMAP 76Csx both lock on to several satellites on the main floor of my house but if I go downstairs the Adafruit maintains lock but the Garmin drops out. The Adafruit usually locks when I'm out and about, in and out of stores, whereas the Garmin often drops out when I'm in a store. I presume the Adafruit has a more recent/better GPS unit.

Pete

michinyon,

Thanks for the input but this is a definately a programming issue. I am very familiar with GPS idiosynchracies and short comings. The string included in my post is the actual output from the GPS device being used. I have 2 terminal windows monitoring both the TX and RX lines of the Arduino and what I see is the command (recieved as a text message) appear on the RX line, the Arduino responds with the AT$GPSACP request to the GPS, The GPS then outputs the GPSACP sentence to the RX line of the arduino, and the arduino does not appear to "see" it.

CHeers
Greg.

zoomkat:
Serial.println("");
Serial.println("GPS Test Example Sketch");
Serial.println("");?

it may sound dumb but I cannot understand line 1 & 3, what will be printed on serial monitor if I write " Serial.println("");"
There is nothing in-between the ""
is it for newline or carriage return??? I suppose Serial.println(); function already has newline or carriage return inbuilt. Kindly correct me if my knowing is wrong.