Polling issue with a Neo-M8N using a Mega2560

Hi All

I have an application where I want to poll a neo-m8n (need data only when requested). I have tried a couple existing libs but the issue seems to be that I have to continually feed the data as opposed to just getting data when requested. The solution I came up with was a combination of reading the GPS using the code from
http://www.monocilindro.com/2016/03/28/reading-gps-data-using-arduino-and-a-u-blox-neo-6m-gps-receiver/ as the base and a tokenizer that I found at CS 221 . It works when I set it up to read the gps using a terminal command 'r'. The main code is as follows (parser and tokenizer seem to be ok):

#include <elapsedMillis.h>
//#include <cstring>
#include <Streaming.h>
#include <math.h>

#define mySerial_GPS Serial2
#define defaultTelemTime 1500
elapsedMillis telem_timer;
#define cout Serial

// Function prototype
char *myStrtok(char *parseStr, char *splitChars);
char    tokChars[8]; // Array of characters used to split on
char    *cptr;       // Pointer to string returned by tokenizers
int     count;       // Counter for words in the string

bool first_loop_exec;

#define GPS_INFO_BUFFER_SIZE 256
char GPS_info_char;
char GPS_info_buffer[GPS_INFO_BUFFER_SIZE];
unsigned int received_char;

uint8_t i; // counter
bool message_started;
uint8_t GPS_mess_no = 0;
uint8_t array_idx = 0;
String gpsdata[25];
bool first_gsa = true;
struct utc {
  int mils;
  int seconds;
  int minutes;
  int hours; 
} datetime;

String op_mode;
int fix, sats_in_use;
String fixtype;
float latlng_age, time_age;
float fix_age, pdop, hdop2, vdop, dop_age, hdop1;
float alt_age, altitude, vVel, sog, cog;
double longitude, latitude;

int val;

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(57600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }
  Serial.println("Connected");
  mySerial_GPS.begin(57600);
  //mySerial_GPS.println("Connected");

  first_loop_exec=true;
  i=0;
  message_started=false;
      telem_timer = 0;
      strcpy(tokChars, ",");
}

void loop() { // run over and over


  //if(telem_timer > defaultTelemTime) {
    if(cout.available() > 0){
      val = cout.read();
    }
    
    if( val == 'r'){
      cout << "request data" << endl;
      read_gps();
      telem_timer = 0;

    }
}


void read_gps(){
   mySerial_GPS.println("$PUBX,00*33"); // data polling to the GPS

  // MANAGES THE CHARACTERS RECEIVED BY GPS
  while (mySerial_GPS.available()) {
    GPS_info_char = mySerial_GPS.read();
    if (GPS_info_char == '

If I change the loop to read the gps every 1 sec for example nothing gets to the parser, changed loop code:

void loop() { // run over and over


  if(telem_timer > defaultTelemTime) {
    //if(cout.available() > 0){
    //  val = cout.read();
    //}
    
    //if( val == 'r'){
      //cout << "request data" << endl;
      read_gps();
      telem_timer = 0;

    }
}

From some debugging it appears that the PUBX message never completes. Not sure what the issue is. If anyone has any suggestions please let me know since this one is driving me crazy.

Mike){ // start of message
      message_started=true;
      received_char=0;
    } else if (GPS_info_char == '*'){ // end of message
      //cout << GPS_mess_no << " ---> " << endl;
      //for (i=0; i<received_char; i++){
      //  Serial.write(GPS_info_buffer[i]); // writes the message to the PC once it has been completely received
      //}
      //Serial.println();

// Make initial call to strtok() passing in the string to be parsed and
    //        the list of characters used to split tokens apart.
    cptr = strtok(GPS_info_buffer, tokChars);
    count = 1; // Initialize the word counter

// Create while() loop to print all the tokens in the string.  Note that
    //    the punctuation has been eliminated leaving just the words from the string.
    //    As long as NULL is passed in as the first argument to strtok it will
    //    continue parsing the last "non-NULL" string passed to it.  It returns
    //    NULL when the entire string has been parsed.
    while(cptr != NULL)
    {
        gpsdata[count] = "";
        for(int len = 0; len < strlen(cptr); len++){
            gpsdata[count] = gpsdata[count]+cptr[len];}
        //cout << "Token " << count << " -->" << gpsdata[count] << "<--\n";
        cptr = strtok(NULL, tokChars); // Get next word
        count++; // Increment counter
    }     
     
      read_pubx();
        val = 0;
      message_started=false; // ready for the new message
     
    } else if (message_started==true){ // the message is already started and I got a new character
      if (received_char<=GPS_INFO_BUFFER_SIZE){ // to avoid buffer overflow
        GPS_info_buffer[received_char]=GPS_info_char;
        received_char++;
      }else{ // resets everything (overflow happened)
        message_started=false;
        received_char=0;
      }
    }

}

while (Serial.available()) {
    mySerial_GPS.write(Serial.read());
  }

}


If I change the loop to read the gps every 1 sec for example nothing gets to the parser, changed loop code:

§DISCOURSE_HOISTED_CODE_1§


From some debugging it appears that the PUBX message never completes. Not sure what the issue is. If anyone has any suggestions please let me know since this one is driving me crazy.

Mike

the issue seems to be that I have to continually feed the data as opposed to just getting data when requested.

I'm not sure what you mean by "feed the data." Are you just saying that you can't poll and parse the response when it comes in? There are several timing issues that could be causing this. Have you disabled all other sentences from being sent?

My NeoGPS library handles generic GPS devices (NMEA) and ublox NEO devices. It parses standard NMEA messages (e.g., $GPRMC), ublox proprietary NMEA messages (e.g., $PUBX) and ublox binary messages (e.g., the UBX protocol). NeoGPS is smaller, faster, more reliable and more accurate than any other library. It can be configured to parse and store just the fields and messages that you really use, saving even more time and space. (LOL, as you know already... I just realized we have had some discussions on github. :smiley: )

The PUBX.ino example program already polls for the proprietary messages. It parses the responses whenever they arrive. Here is a NeoGPS version of your sketch:

#include "ubxNMEA.h"

#include <Streaming.h>
#include <math.h>

#define cout Serial
#define gps_port Serial2
#define defaultTelemTime 1500
uint32_t telem_timer;

ubloxNMEA gps;
gps_fix   fix;

// Use a Finite-state machine (aka FSM) to implement the
//   periodic request of a "$PUBX,00# message.
enum state_t { WAITING_TO_REQ, REQUESTING_PUBX };  // possible states
state_t state;                                     // current state variable

void setup() {

  // Open serial communications and wait for port to open:
  Serial.begin(57600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }
  cout << F("Connected") << endl;
  
  gps_port.begin(57600);

}

void loop() { // run over and over

//if (millis() - telem_timer > defaultTelemTime) {
  if (cout.available() > 0){
    char val = cout.read();
  
    if ((val == 'r') && (state == WAITING_TO_REQ)) {
      cout << F("request data") << endl;
      gps.send_P( &gps_port, F("PUBX,00") ); // data polling to the GPS
      state        = REQUESTING_PUBX;
      telem_timer  = millis();
    }
  }

  //  This runs all the time to keep the port empty.  It only uses the
  //    the returned fix structure when we're in the REQUESTING state.
  //  Normally, there shouldn't be anything to process.

  while (gps.available( gps_port )) {
    fix   = gps.read();

    if (state == REQUESTING_PUBX) {
      state = WAITING_TO_REQ; // got it!
      displayGPS();
    }
  }

  //  If we didn't get a response within 3 seconds, warn the console
  
  if ((state == REQUESTING_PUBX) && (millis() - telem_timer > 3000L)) {
    cout << F("No GPS response!") << endl;
    state = WAITING_TO_REQ; // go back to waiting
  }

} // loop

void displayGPS()
{
  cout << F("Fix: ");
  if (fix.valid.status)
    cout << (uint8_t) fix.status;
  cout << endl;

  cout << F("Sats: ");
  if (fix.valid.satellites)
    cout << fix.satellites;
  cout << endl;

  cout << F("UTC: ");
  if (fix.valid.date || fix.valid.time) {
    cout << fix.dateTime << '.';
    if (fix.dateTime_cs < 10)
      cout << '0';
    cout << fix.dateTime_cs;
  }
  cout << endl;

  cout << F("Loc: ");
  if (fix.valid.location)
    cout << fix.latitudeL() << ',' << fix.longitudeL();
  else
    cout << endl;
  cout << endl;

  cout << F("COG: ");
  if (fix.valid.heading)
    cout << fix.heading_cd();
  cout << endl;

  cout << F("SOG: ");
  if (fix.valid.speed)
    cout << fix.speed_mkn();
  cout << endl;

  cout << F("Alt: ");
  if (fix.valid.altitude)
    cout << fix.altitude_cm();
  cout << endl;

  cout << F("DOP (h,v,p): ");
  if (fix.valid.hdop)
    cout << fix.hdop;
  cout << ',';

  if (fix.valid.vdop)
    cout << fix.vdop;
  cout << ',';

  if (fix.valid.pdop)
    cout << fix.pdop;
  cout << endl;

  cout << endl;

  // Your variables that were not printed:
  //    float latlng_age, time_age, fix_age, hdop2, dop_age, hdop1;
  //       (The ages should be 1 or 2 seconds, at most.)

} // displayGPS

I have attached two modified config files (replace the github versions with these).

Be aware that there are some issues with the polling approach (related discussions here and here). Specifically, are you powering down the GPS? Also, there are many lurking issues with String. o_O

Cheers,
/dev


Notes on the sketch:

* Timers are easily implemented with a timestamp. The comparison is on an interval, which correctly handles the millis() rollover problem.

* This shows the FSM technique for dealing with multiple "things" that need to be done at the same time. The 'r' command is only parsed when the state is WAITING, and the PUBX,00 response is only processed when the state is REQUESTING. MORE IMPORTANTLY, it does not stop everything else while the poll command goes out (2ms) and the reply comes back (up to 2 seconds!).

* The $PUBX,00 message does not include the date. If you need the date, you would need to enable another message. You could use an RMC, ZDA or PUBX,04, according to this table.

* I updated ubxNMEA.h and .cpp on github to add the VDOP field, an oversight on my part. Those two files also need to be in your sketch directory, just like the rest of the NeoGPS files.

* I just noticed that the $PUBX,00 has a TDOP field, not a PDOP field. Table corrected. There is no TDOP field in gps_fix yet, although I doubt you really need that.

* There was a trick to getting NeoGPS to use the $PUBX,00 as the LAST_SENTENCE_IN_INTERVAL. See NMEAGPS_cfg.h, line 29. This technique allows you to enable standard messages if you want, without disturbing the INTERVAL. You may want to confirm the sentence order if coherency is important.

* I wasn't sure what you are doing with the variables, so I just printed them out with the displayGPS() routine. It shows you how to access the variables in the fix structure (see also Data Model page). It's really a snippet from Streamers.cpp.

* I'd be curious about the current sketch size vs. the NeoGPS size. I think you'll find this version is quite a bit smaller. If not, I'll eat my hat. :smiley:

GPSfix_cfg.h (956 Bytes)

NMEAGPS_cfg.h (10.9 KB)

Hi /dev

Thanks for providing the example. Unfortunately it does not work. It keeps returning a No GPS Response. It is about 1k smaller in size than my sketch so that is good news. I did download the latest version of NeoGPS from GitHub as well as replacing the two attached files. As a double check I did run my sketch again just to make sure the GPS was responding. I will have to look at it further.

In going through your table I noticed that you lat, long, alt errors as not available. However, there is a hack and vAcc available. As you said PDOP is not available.

Just to let you know I am working on a autonomous rover that has several modes of operation: manual, obstacle avoidance, RC and waypoint nav. When the RC mode is selected telemetry is sent back to the by the radio link on serial3 to the pc which includes GPS coordinates every second. In waypoint nav I check my position (distance and direction) against the waypoint every 1 second. I tried just letting the GPS stream the data to the serial connection and read it only when necessary but ran into a strange problem where the lat and long would never update or would never read (think I had a overflow). I did try your NeoHWSerial as a test but still had problems. All modes work except for the GPS issue.

If you have any other ideas I am open.

The settings for the GPS has a update rate of 0.2 seconds.

By the way its a great library with a lot of flexibility.

Thanks for your help.

Unfortunately it does not work. It keeps returning a No GPS Response.

Then the NEO-M8 is really not replying with a $PUBX,00 message. My NEO-6M does reply.

After looking at the spec, I think that the NEO-M8 may not support the query message "$PUBX,00". Whut? There's no description of any kind of NMEA message that can request the "$PUBX,00" proprietary message response. There is a UBX binary message that can set the rate, and you can poll for the UBX binary messages, but there's nothing about the NMEA proprietary text messages. :frowning: It's not even documented as "deprecated".

However, you say:

As a double check I did run my sketch again just to make sure the GPS was responding. I will have to look at it further.

Are you sure it's replying with a "$PUBX,00", and that it isn't constantly sending the "$PUBX,00"? You can configure the NEO-M8 to send that sentence periodically.

I had to disable all other messages, because they were filling out the fix structure as if a "$PUBX,00" had been received.

In waypoint nav I check my position (distance and direction) against the waypoint every 1 second. I tried just letting the GPS stream the data to the serial connection and read it only when necessary but ran into a strange problem where the lat and long would never update or would never read (think I had a overflow).

That problem is very common. If you show us the code, we could try to help. I suspect it is a blocking routine like readGPS. Notice how the NeoGPS sketch never stays in a while loop, waiting for a response. It just sends the query, changes the state variable, and continues. Then it checks available over and over, thousands of times. When the GPS device eventually finishes sending the reply, it is handled (converted to a fix and printed) and causes a state transition (back to WAITING).

Calculating the distance and direction to a waypoint is fairly fast, and should not cause any GPS data loss. Printing those results 5 times per second could cause data loss.

Cheers,
/dev

After reviewing several of your repositories, I'd like to point you to this again. The key to programming in this environment is to structure your loop to constantly "visit" each thing that you might be doing.

This is very different from the typical desktop coding, where the program is structured very linearly. Do this, then do that. In the embedded environment, you have to check this, check this, check this, do that, check this... Never use delay, and never stay in a while loop, waiting for data.

Instead, use state variables to "remember" what things are active. When you visit each "thing", the state variable helps you decide what to do.

For example, the GPS loop must be constantly called from loop, reading fixes when they are available. If no fixes are available, no big deal. If fixes are available, just save them in a fix variable. Use the fix immediately if you need to, or let some other part of your program access it as a "current location".

You will also need an IMU loop. Maybe it checks the FIFO and reads out the values. Again, use them immediately if you need to, otherwise save them in a variable for some other part of your program to use. Set a flag to indicate that new data is available if it helps other parts of your program detect the "newness".

Ultimately, your loop() function should just call these subordinate loops. Most of the time, the subordinate loops don't have anything to do. Sometimes, an event occurs and the subordinate loops will get to do a little work. Don't take too long, or the other loops will drop some dishes. :wink:

Cheers,
/dev

Hi /dev

Then the NEO-M8 is really not replying with a $PUBX,00 message

Positive that the NEO-M8 supports the $PUBX,00, it is in the spec that I have also, I the output from my sketch showed that is receiving the message along with the parsed info:

PUBX,00,114201.10,4046.49941,N,07348.88331,W,5.833,G3,12,8.4,0.163,167.71,0.036,,9.37,4.68,4.23,4,0,0

G3,4, 11:42:1.9,40.774997, -73.814712, 9.37, 0.163, 167.710,5.83

As always your responses are right on target. I did isolate the problem and get my sketch working. Not as elegant as yours, which I need to get into, along with state machines. The fix was to put an extra loop into the read_gps just like you suggested to keep the whole gps process in the gps module. I did test it in the main code and it worked this time around, not as good as I wanted. The issue I am running into now is on a mega double and float are the same size and really need it to be double. Looking at a way around that one.....

Here is the updated code that I am using:

#include <elapsedMillis.h>
//#include <cstring>
#include <Streaming.h>
#include <math.h>

#define mySerial_GPS Serial2
#define defaultTelemTime 1000
elapsedMillis telem_timer;
#define cout Serial

// Function prototype
char *myStrtok(char *parseStr, char *splitChars);
char    tokChars[8]; // Array of characters used to split on
char    *cptr;       // Pointer to string returned by tokenizers
int     count;       // Counter for words in the string

bool first_loop_exec;

#define GPS_INFO_BUFFER_SIZE 256
char GPS_info_char;
char GPS_info_buffer[GPS_INFO_BUFFER_SIZE];
unsigned int received_char;

uint8_t i; // counter
bool message_started = false;
uint8_t GPS_mess_no = 0;
uint8_t array_idx = 0;
String gpsdata[25];
bool first_gsa = true;
struct utc {
  int mils;
  int seconds;
  int minutes;
  int hours; 
} datetime;

String op_mode;
int fix, sats_in_use;
String fixtype;
float latlng_age, time_age;
float fix_age, pdop, hdop2, vdop, dop_age, hdop1;
float alt_age, altitude, vVel, sog, cog;
double longitude, latitude;

int val;

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(57600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }
  Serial.println("Connected");
  mySerial_GPS.begin(57600);
  //mySerial_GPS.println("Connected");


  first_loop_exec=true;
  i=0;
  message_started=false;
      telem_timer = 0;
      strcpy(tokChars, ",");
}

void loop() { // run over and over
  if (first_loop_exec == true){
    delay(2000);
    mySerial_GPS.println(F("$PUBX,40,RMC,0,0,0,0*47")); //RMC OFF
    delay(100);
    mySerial_GPS.println(F("$PUBX,40,VTG,0,0,0,0*5E")); //VTG OFF
    delay(100);
    mySerial_GPS.println(F("$PUBX,40,GGA,0,0,0,0*5A")); //CGA OFF
    delay(100);
    mySerial_GPS.println(F("$PUBX,40,GSA,0,0,0,0*4E")); //GSA OFF
    delay(100);
    mySerial_GPS.println(F("$PUBX,40,GSV,0,0,0,0*59")); //GSV OFF
    delay(100);
    mySerial_GPS.println(F("$PUBX,40,GLL,0,0,0,0*5C")); //GLL OFF
    delay(1000);
    mySerial_GPS.println(F("$PUBX,40,VLW,0,0,0,0*06")); //vlw OFF
    delay(1000);
    first_loop_exec = false;
  }

  if(telem_timer > defaultTelemTime) {
    //if(cout.available() > 0){
    //  val = cout.read();
    //}
    
    //if( val == 'r'){
      mySerial_GPS.write("$PUBX,00*33\r\n"); // data polling to the GPS
      read_gps();
      telem_timer = 0;
      val = 0;
      count = 1;
    }
    
}


void read_gps(){
[b]while(count < 21){[/b]
  // MANAGES THE CHARACTERS RECEIVED BY GPS
  while (mySerial_GPS.available()) {
    GPS_info_char = mySerial_GPS.read();
    //cout << GPS_info_char << endl;
    if (GPS_info_char == '

Still have a lot of work to do on this and getting more into your lib. Also, there is a problem with TinyGPS++ in that it only supports GPS messages not GLONASS (GP vs GN).

Cheers
Mike){ // start of message
      message_started = true;
      received_char=0;
    } else if (GPS_info_char == '*'){ // end of message
      for (i=0; i<received_char; i++){
        Serial.write(GPS_info_buffer[i]); // writes the message to the PC once it has been completely received
      }
      Serial.println();

// Make initial call to strtok() passing in the string to be parsed and
    //        the list of characters used to split tokens apart.
    cptr = strtok(GPS_info_buffer, tokChars);
    count = 1; // Initialize the word counter

// Create while() loop to print all the tokens in the string.  Note that
    //    the punctuation has been eliminated leaving just the words from the string.
    //    As long as NULL is passed in as the first argument to strtok it will
    //    continue parsing the last "non-NULL" string passed to it.  It returns
    //    NULL when the entire string has been parsed.
    while(cptr != NULL)
    {
        gpsdata[count] = "";
        for(int len = 0; len < strlen(cptr); len++){
            gpsdata[count] = gpsdata[count]+cptr[len];}
        //cout << "Token " << count << " -->" << gpsdata[count] << "<--\n";
        cptr = strtok(NULL, tokChars); // Get next word
        count++; // Increment counter
    }
   
      read_pubx();
      val = 0;
      message_started=false; // ready for the new message
     
    } else if (message_started==true){ // the message is already started and I got a new character
      if (received_char<=GPS_INFO_BUFFER_SIZE){ // to avoid buffer overflow
        GPS_info_buffer[received_char]=GPS_info_char;
        received_char++;
      }else { // resets everything (overflow happened)
        message_started=false;
        received_char=0;
      }
    }

}

}
}


Still have a lot of work to do on this and getting more into your lib. Also, there is a problem with TinyGPS++ in that it only supports GPS messages not GLONASS (GP vs GN).

Cheers
Mike

Positive that the NEO-M8 supports the $PUBX,00, it is in the spec that I have also

R11 of the spec says the NEO-M8 can output the "$PUBX,00", but it does not say anything about polling for it. I guess it's just an undocumented feature, probably for backwards compatibility.

I have some ideas to try, but I'll post them later...

The issue I am running into now is on a mega double and float are the same size and really need it to be double.

Are you needing more precise locations, distances or bearings?

Ok /dev. Thanks for looking at this. The answer to your last ion is yes. Right now looking at a couple of options. Found a repository where someone is using math and float libs to to do that or using the bignumber lib and the bignumber math. Both options are going to increase the size of the sketch. Don't know how to get distance and bearings using long ints.

Thanks again for your help.
mike

Don't know how to get distance and bearings using long ints.

Rats. If only NeoGPS could do that...

Oh wait, it does. :smiley: See the Location documentation and the NMEAaverage example. The example sketch calculates a super-accurate stationary location by averaging thousands of fixes. It also shows how to calculate distance and bearing.

These calculations are more accurate than the GPS locations being reported, within millimeters at short distances. You could use a "big number" library, or look for a double implementation, but it wouldn't be any better. Just bigger and slower.

The NMEAloc example shows how to print the long ints as if they were high-precision floats.

Anything else you need? :slight_smile:

Cheers,
/dev

P.S. I've almost got a sketch for you to try for the PUBX poll issue above.

OK now I am confused. When I look at location.cop it looks line you dart out using integer posits multiplied by scannng factor then you do floatingpoint math to calculate bearingand distance as floats. Would have thought once you did this you would loose the precion from the long and lat. That was why I was looking at using essentially doublesusing the two libs mentioned. Not an expert so could be wrong. Understand about averagingand was going to give a try eventually. Just wanted to get this working first.

Believe me would rather use your lib.. looking forward to the new sketch

Mike

PS. This is the link I was referring to for 64 bit distance and bearing and calculations.

Ok, there was a problem when the NMEA config had all sentences disabled and ubxNMEA had PUBX 00 and 04 enabled. I had disabled 04 in my build, so I didn't see the error. Thanks! There's a new version of ubxNMEA.h that you need, and here's an updated sketch:

#include "ubxNMEA.h"

#include <Streaming.h>
#include <math.h>

#define cout Serial
#define gps_port Serial2
#define defaultTelemTime 1500
uint32_t telem_timer;

ubloxNMEA gps;
gps_fix   fix;

// Use a Finite-state machine (aka FSM) to implement the
//   periodic request of a "$PUBX,00# message.
enum state_t { WAITING_TO_REQ, REQUESTING_PUBX };  // possible states
state_t state;                                     // current state variable

//---------------------------------------------------------

void setup() {

  Serial.begin(57600);
  while (!Serial)
    ;

  cout << F("Connected") << endl;

  gps_port.begin(57600);

  configMessages( false );
}

//---------------------------------------------------------

void loop() {

  // bool timeToPoll = (millis() - telem_timer > defaultTelemTime);
  bool timeToPoll = false;

  // Are there any commands from the Serial Monitor?
  
  if (cout.available() > 0){
    char cmd = cout.read();
  
    if (cmd == 'x') {
      // Toggle all the other messages on or off
      static bool onOff = false;
      onOff = !onOff; // toggle
      configMessages( onOff );

    } else if (cmd == 'r') {
      if (state == WAITING_TO_REQ)
        timeToPoll = true;
    }
  }

  // Is it time to request a PUBX,00 message?

  if (timeToPoll) {
    cout << F("request data") << endl;
    gps.send_P( &gps_port, F("PUBX,00") ); // data polling to the GPS
    state        = REQUESTING_PUBX;
    telem_timer  = millis();
  }

  //  Is there any data from the GPS?

  while (gps.available( gps_port )) {
    fix   = gps.read();

    if (state == REQUESTING_PUBX) {
      state = WAITING_TO_REQ; // got it!
      displayGPS();
    }
  }

  //  Did we timeout getting a PUBX response?
  
  if ((state == REQUESTING_PUBX) && (millis() - telem_timer > 3000L)) {
    cout << F("No GPS response!") << endl;
    state = WAITING_TO_REQ; // go back to waiting
  }

} // loop

//---------------------------------------------------------

void configMessages( bool on )
{
  if (on) {
    gps.send_P( &gps_port, F("PUBX,40,RMC,0,1,0,0")); //RMC ON
    gps.send_P( &gps_port, F("PUBX,40,VTG,0,1,0,0")); //VTG ON
    gps.send_P( &gps_port, F("PUBX,40,GGA,0,1,0,0")); //CGA ON
    gps.send_P( &gps_port, F("PUBX,40,GSA,0,1,0,0")); //GSA ON
    gps.send_P( &gps_port, F("PUBX,40,GSV,0,1,0,0")); //GSV ON
    gps.send_P( &gps_port, F("PUBX,40,GLL,0,1,0,0")); //GLL ON
    gps.send_P( &gps_port, F("PUBX,40,VLW,0,1,0,0")); //vlw ON
  } else {
    gps.send_P( &gps_port, F("PUBX,40,RMC,0,0,0,0")); //RMC OFF
    gps.send_P( &gps_port, F("PUBX,40,VTG,0,0,0,0")); //VTG OFF
    gps.send_P( &gps_port, F("PUBX,40,GGA,0,0,0,0")); //CGA OFF
    gps.send_P( &gps_port, F("PUBX,40,GSA,0,0,0,0")); //GSA OFF
    gps.send_P( &gps_port, F("PUBX,40,GSV,0,0,0,0")); //GSV OFF
    gps.send_P( &gps_port, F("PUBX,40,GLL,0,0,0,0")); //GLL OFF
    gps.send_P( &gps_port, F("PUBX,40,VLW,0,0,0,0")); //vlw OFF
  }
}

//---------------------------------------------------------

void displayGPS()
{
  cout << F("Fix: ");
  if (fix.valid.status)
    cout << (uint8_t) fix.status;
  cout << endl;

  cout << F("Sats: ");
  if (fix.valid.satellites)
    cout << fix.satellites;
  cout << endl;

  cout << F("UTC: ");
  if (fix.valid.date || fix.valid.time) {
    cout << fix.dateTime << '.';
    if (fix.dateTime_cs < 10)
      cout << '0';
    cout << fix.dateTime_cs;
  }
  cout << endl;

  cout << F("Loc: ");
  if (fix.valid.location)
    cout << fix.latitudeL() << ',' << fix.longitudeL();
  else
    cout << endl;
  cout << endl;

  cout << F("COG: ");
  if (fix.valid.heading)
    cout << fix.heading_cd();
  cout << endl;

  cout << F("SOG: ");
  if (fix.valid.speed)
    cout << fix.speed_mkn();
  cout << endl;

  cout << F("Alt: ");
  if (fix.valid.altitude)
    cout << fix.altitude_cm();
  cout << endl;

  cout << F("DOP (h,v,p): ");
  if (fix.valid.hdop)
    cout << fix.hdop;
  cout << ',';

  if (fix.valid.vdop)
    cout << fix.vdop;
  cout << ',';

  if (fix.valid.pdop)
    cout << fix.pdop;
  cout << endl;

  cout << endl;

} // displayGPS

I added an 'x' command to toggle all the other messages on and off, in a handy configMessages routine. They are turned off in setup.

Also notice that timeToPoll can be periodic if you switch which declaration is commented (line 38 or 39). It is turned off above... you have to type the 'r' command.

Cheers,
/dev

OK now I am confused. When I look at location.cpp it looks line you start out using integer positions multiplied by scaling factor then you do floating point math to calculate bearing and distance as floats. Would have thought once you did this you would loose the precision from the long and lat.

The problem for most libraries is that the distance calculations use float for the lat and long values: they only have 7 significant digits. When you subtract two latitudes to get a difference (needed for various calculations), the result is meaningless when the latitudes differ in the 8th significant digit (i.e., they're close).

For example, the original "4046.49941,N" from the GPS has 9 significant digits (almost).

NeoGPS converts that to 407749902, the long int version of the floating-point value 40.7749901667. Notice that the long int still has 9 significant digits. The float value would have been 40.77499 and would have lost two digits.

To calculate distance or bearing, two values are subtracted. For example, NeoGPS would convert "4046.49942,N" to 407749903, and subtracting those two numbers would give the correct answer of 1 (scaled by 107). If it had been stored as a float, the difference would have been 0. :frowning:

And that 1 can be converted to a float with value 0.0000001 with no loss of precision (it only had 1 significant digit). And there's very little precision lost after taking sines and cosines of those deltas: the calculated sine of 0.000000100000 is still very accurate. So when the locations are close (within a few degrees), NeoGPS calculations are very good.

There is some precision lost when the original lat or long (not a difference) is used in a float calculation. Fortunately, it is significant only when the points are far apart (10's of degrees). I would claim that it usually doesn't matter because (1) you usually don't care about millimeters over 100's of km, and (2) the ellipsoidal calculation is an only an approximation of the Earth's surface. Slight changes in altitude or course deviations will introduce larger errors. And we are talking about Arduinos, after all. :wink:

FWIW, I have compared the results with Google Maps and this excellent site, which only gives 4 significant digits for the reasons stated above.

Cheers,
/dev

/dev

Downloaded your library again and the sketch you provided works like a charm. Its actually about 300 bytes smaller than your previous sketch and responds to the read command quickly. Probably a great sketch to add to your library if you want.

Thanks for thanking the time to explain to me the distance and bearing accuracy. Very detailed as all your explanations are and very informative. Always learning something. Looks like that should solve my last issue.

Now I have to get it incorporated into my master sketch and get it working in conjunction with other elements of the rover. Will keep you posted on how that goes. That may take a few days to get accomplished as I just had a family emergency. Doing this right now just to keep my mind busy. :slight_smile:

thanks a bunch for putting the time and effort into developing this for me. Can't tell you how much I appreciate it.

Mike

Hi /dev.

Running some tests using your sketch and incorporating it into the Rover sketch. I think I need a better understanding of what is happening. I reduced your sketch to a minimum and turned off all messages in setup. The following runs and returns GPS data as your original sketch:

//---------------------------------------------------------

void loop() {
  // Are there any commands from the Serial Monitor?

  if (cout.available() > 0){
    char cmd = cout.read();
    if (cmd == 'r') {
      cout << F("request data") << endl;
      gps.send_P( &gps_port, F("PUBX,00") ); // data polling to the GPS
    }
  }

  //  Is there any data from the GPS?
  while (gps.available( gps_port )) {
    fix  = gps.read();
    displayGPS();
  }

} // loop

I only included the loop to keep it simple as that is where the problem is. Now when I change it to:

void loop() {
  // Are there any commands from the Serial Monitor?

  if (cout.available() > 0){
    char cmd = cout.read();
    if (cmd == 'r') {
      cout << F("request data") << endl;
      gps.send_P( &gps_port, F("PUBX,00") ); // data polling to the GPS

      //  Is there any data from the GPS?
      while (gps.available( gps_port )) {
          fix  = gps.read();
          displayGPS();
      }
    }
  }
} // loop

Please help me in understanding. I would have expected that when you sent the poll command it would return the data almost immediately unless I need to add a test within the while(gps.available ...) for timeout?

thanks
Mike

I reduced your sketch to a minimum... The following runs:

void loop()

{
 if (command chars available) {
   if ('r' command) {
     request GPS sentence;
   }
 }

while (GPS sentence available)
   displayGPS();



Now when I change it to:



void loop()
{
 if (command chars available) {
   if ('r' command) {
     request GPS sentence;
     while (GPS sentence available)
       displayGPS();
   }
 }



... it doesn't work. Why?

The latter version expects demands a certain order of events: after sending the request, it reads the response. If it doesn't happen in that order, immediately, it doesn't work.

The former version checks for command chars (there may not be any) and checks for GPS fixes (there may not be any). 99,999 times out of 100,000, there is nothing to do. But once in a great while (on the Arduino time scale), a command character is received. That command character causes a poll command string (13 characters) to be sent to the GPS, and that takes ~2ms. While those characters are gradually trickling out of the Arduino, loop executes thousands of times. Thousands! When the GPS device finally receives those characters, it puts the "$PUBX,00" response on its TO DO list.

The GPS device has a mind of its own, and it will only respond when it's good and ready. Similarly, the Arduino UART will let you have the received character when it's good and ready. The same is true of all the devices you use, whether they are inside the Arduino (e.g., the UART) or outside (e.g., an IMU).

You have configured the GPS device to provide fixes at 5Hz, or every 200ms. Between each fix, the Arduino will keep checking (thousands of times) for a new fix, and there won't be any available (ditto for command chars, IMU updates, etc.). The GPS device only processes its TO DO list once every 200ms. So, if you're lucky, the GPS device will formulate a "$PUBX,00" response for the next update, but that could be up to 200ms after you asked for it. That is hundreds of thousands of loop iterations.

If you're not lucky, you may have missed the update interval, and the response won't be formulated until almost 400ms after you asked for it. 400ms is an eternity to the Arduino.

When the GPS device finally does the things on its TO DO list, it queues up the ~112 response characters, and they begin trickling back to the Arduino. At 57600, it will be ~20ms before the last character is transmitted.

As each of those "$PUBX,00" characters are received by the Arduono, the UART tells the CPU that a character is ready, with an interrupt. The Arduino software handles the interrupt and stores the received character in an input buffer. When loop calls gps.available( gps_port ), that character is read by NeoGPS and parsed into an internal fix structure. This could be as long as ~422ms after you called gps.poll. That's why your expectation is wrong:

I would have expected that when you sent the poll command it would return the data almost immediately

Nope. Another way to look at this is that your Arduino is running 16 million instructions per second (it's actually less than that because many instructions take more than 1 clock cycle), and the serial port is sending only 57600 bits per second, which is actually 5760 characters per second. Please compare 16 million to 5760. :o It's not "immediate" by a long shot.

The key concept is that you can't write the sketch as if these events (e.g., character received) are going to happen when you want, in the order you want. All you can do is check to see if they have happened. That is called "polling". You have to make sure that you poll frequently enough that something doesn't get lost.

Usually, the order that you poll is immaterial. For example, most of the checks in the NeoGPS sketch could be rearranged. But the checks cannot be nested to force a certain order. And if you try to block at a certain point with a delay or a nested while loop, the Arduino can't do any of the other things it's supposed to be doing. You will be wasting millions of instruction times doing nothing. Your rover will have many things to do, so you can't be wasting time like that. Many libraries will not work if you try to structure your program like that. You must

    check A, check B, check C, check D, 
    check A, check B, [b]do C[/b], check D,
    check A, check B, check C, ...

Cheers,
/dev
who has to go check a few things instead of waiting here for a response... :wink:

/dev

Thanks for your lengthy and very informative post. Based on your comments I did get it working in a manner that will integrate nicely into my rover program. For reference in case any one else is interested follows:

#include "ubxNMEA.h"

#include <Streaming.h>
#include <math.h>

#define cout Serial
#define gps_port Serial2
#define defaultTelemTime 1500
uint32_t telem_timer;
int count;

ubloxNMEA gps;
gps_fix   fix;

// Use a Finite-state machine (aka FSM) to implement the
//   periodic request of a "$PUBX,00# message.
enum state_t { WAITING_TO_REQ, REQUESTING_PUBX }; // possible states
state_t state;                                    // current state variable

//---------------------------------------------------------

void setup() {

  Serial.begin(57600);

  cout << F("Connected") << endl;

  gps_port.begin(57600);

    gps.send_P( &gps_port, F("PUBX,40,RMC,0,0,0,0")); //RMC OFF
    gps.send_P( &gps_port, F("PUBX,40,VTG,0,0,0,0")); //VTG OFF
    gps.send_P( &gps_port, F("PUBX,40,GGA,0,0,0,0")); //CGA OFF
    gps.send_P( &gps_port, F("PUBX,40,GSA,0,0,0,0")); //GSA OFF
    gps.send_P( &gps_port, F("PUBX,40,GSV,0,0,0,0")); //GSV OFF
    gps.send_P( &gps_port, F("PUBX,40,GLL,0,0,0,0")); //GLL OFF
    gps.send_P( &gps_port, F("PUBX,40,VLW,0,0,0,0")); //vlw OFF
}

//---------------------------------------------------------

void loop() {
  // Are there any commands from the Serial Monitor?

  if (cout.available() > 0){
    char cmd = cout.read();
    if (cmd == 'r') {
      cout << F("request data") << endl;
      gps.send_P( &gps_port, F("PUBX,00") ); // data polling to the GPS
      state = REQUESTING_PUBX;

      //  Is there any data from the GPS?
      while(state == REQUESTING_PUBX){
        while (gps.available( gps_port )) {
          fix  = gps.read();
          if (state == REQUESTING_PUBX) {
            state = WAITING_TO_REQ; // got it!
            displayGPS();
          }
        }
      }
    }
  }
} // loop


//---------------------------------------------------------

void displayGPS()
{
  cout << F("Fix: ");
  if (fix.valid.status)
    cout << (uint8_t) fix.status;
  cout << endl;

  cout << F("Sats: ");
  if (fix.valid.satellites)
    cout << fix.satellites;
  cout << endl;

  cout << F("UTC: ");
  if (fix.valid.date || fix.valid.time) {
    cout << fix.dateTime << '.';
    if (fix.dateTime_cs < 10)
      cout << '0';
    cout << fix.dateTime_cs;
  }
  cout << endl;

  cout << F("Loc: ");
  if (fix.valid.location)
    cout << fix.latitudeL() << ',' << fix.longitudeL();
  else
    cout << endl;
  cout << endl;

  cout << F("COG: ");
  if (fix.valid.heading)
    cout << fix.heading_cd();
  cout << endl;

  cout << F("SOG: ");
  if (fix.valid.speed)
    cout << fix.speed_mkn();
  cout << endl;

  cout << F("Alt: ");
  if (fix.valid.altitude)
    cout << fix.altitude_cm();
  cout << endl;

  cout << F("DOP (h,v,p): ");
  if (fix.valid.hdop)
    cout << fix.hdop;
  cout << ',';

  if (fix.valid.vdop)
    cout << fix.vdop;
  cout << ',';

  if (fix.valid.pdop)
    cout << fix.pdop;
  cout << endl;

  cout << endl;

} // displayGPS

v/R
Mike

Well, I really tried...

For anyone that follows, don't use that loop structure. It is not "based on my comments"; it is the opposite of my comments. Spinning dishes! That sketch focuses on one dish.

By nesting the while loop, nothing else will happen until a GPS fix arrives: no commands, no IMU, no other serial devices, nothing, perhaps for almost 500ms. It doesn't need a state variable because the a certain order is forced. And if the GPS quits working, that code will wait forever.

The sketch in reply #11 performs the exact same thing, but it does not block, will not hang and will easily accommodate other "tasks".

By nesting the while loop, nothing else will happen until a GPS fix arrives: no commands, no IMU, no other serial devices, nothing, perhaps for almost 500ms. It doesn't need a state variable because the a certain order is forced. And if the GPS quits working, that code will wait forever.

So far if no data is available and I hit a r it returns no data. /dev's code is the best way to implement if everything is controlled with the Arduino loop structure which includes reading an IMU etc. with all calls returning to the loop. In my particular application it is not set up that way and I have essentially independent loops for particular functions which includes rc and waypoint navigation which is reading interrupts, imu, encoders etc. So far in static testing it is working beautifully with no locks ups - lucky I guess. If nothing available it returns blanks or zeros in my case.

/dev is also correct I did not implement a state machine but only used it create the second loop to get it to read when the command to read is implemented and yes for my application I was focusing on a single dish. If I had to rover software over I would do it differently using /dev's approach and use a behavior based approach to obstacle avoidance. Maybe in the next life.

By the way NeoGPS is the way to go if you are planning to use a GPS. While TinyGPS is good the stock version does not read Glosnass only GPS unless you use a modified library.

My thanks to /dev.