Go Down

Topic: 20Hz GPS (115200 serial) and serial parsing  (Read 428 times) previous topic - next topic

fuzion

I am old enough to know that my opinions about me and my work don't really count for anything. (Including this one)

...R
man i am sorry for any misunderstandings here i didn't mean anything. i respect everyone's work especially if his guidelines point to a better code. and what i posted is indeed without proper coding.

fuzion

*   I have attached a NeoGPS version of your sketch.  I changed all the double-quoted strings to use the F macro (saves RAM), but I wasn't sure if your Digole library supports that.  You'll get a compile error about "FlashStringHelper" if it doesn't.
ok i did a first test, it works out of the box (surprized) at 1 update per second on screen.

again there are quickly 2 issues:

1.  the data in (fixCount) does not always increment by 20. when i do the split, it drops to adding 16-18 per second and stays there, sometimes lower. if i make the refresh quicker, it drops to even 1 per second.

2. the split is always in the 0.100 accuracy. this happened in my code also, which should be there in 0.050. The first split is accurate to the milisecond, but every next one has 0.1 resolution (the laptime miliseconds i see are always like .200 .199 .300 .901 .000). this is weird.

anyway i will try with neogps from scratch (using more functions this time :D ) and i emailed sparkfun for a firmware

-dev

Quote from: fuzion
if... what i posted is indeed without proper coding.
Robin2 was trying to point out that your loop was too long and needed to be broken up into smaller parts.  See attached.

Quote from: fuzion
it works out of the box (surprized)
I tried it on your system while you weren't looking.

Quote from: fuzion
- do you think that the I2C protocol for the screen messes with the serial somehow?...

1.  the data in (fixCount) does not always increment by 20. when i do the split, it drops to adding 16-18 per second and stays there, sometimes lower. if i make the refresh quicker, it drops to even 1 per second.
This means the display update takes too long.  Here are a few things to try:

1)  Reduce the display update time by drawing the labels during setup and only printing the variables in loop.  You don't have to set the cursor position for every character, so that would reduce the number of commands sent to the display (see attached).

2) Use the Interrupt-Driven Approach and a FIX_MAX of 2 or more.  This will buffer the fixes while you are updating the display (yes, concurrently).

3) Use SPI @ 8Mbps, which is much faster than I2C @ 400kbps.  Serial can easily go to 500kbps, sometimes higher.

You should do (1), regardless.  You could try (2) without changing your hardware.

Quote from: fuzion
2. the split is always in the 0.100 accuracy.
You only check the splitsim pin when a new fix is available.  Pull it out of the "if gps.available" block if you want to constantly check it (see attached).

Quote from: fuzion
the laptime miliseconds i see are always like .200 .199 .300 .901 .000.  this is weird.
There is a slight variability in the GPS device and the time that the Arduino handles each character interrupt.  You will also see the times "creep", because the Arduino crystal is not exactly 16MHz.

Cheers,
/dev

fuzion

wow.

neogps is indeed faster, and i used the NMEA_isr.ino from the beginning with interrupts configured. the serial monitor is flawless at 50ms intervals no matter what i do with the screen. a small issue: the starttime=millis() now gives starttime the value 1108882091 but millis right after that go to zero normally.

i dont know if i will try the spi, perhaps later on, as i dont care anymore :D

thanks man this was great.

-dev

Quote
a small issue: the starttime=millis() now gives starttime the value 1108882091 but millis right after that go to zero normally.
We'd have to see your sketch...

fuzion

Code: [Select]
#include <Arduino.h>
#include <NMEAGPS.h>

#define _Digole_Serial_I2C_
#include <DigoleSerial.h>
#include <Wire.h>
DigoleSerialDisp mydisp(&Wire, '\x27');

int i, j, k, l = 0, m, n, o, p, q, r, x, y, split = 0, touchinput = 23, splitsim = 30,  miliseconds = 0,
             seconds0 = 0, seconds1 = 0, minutes = 0;

const size_t MAX_LAPTIMES = 100;
long  laptime = 0, startlap = 0, lastlap = 0, laptimes[ MAX_LAPTIMES ];

//======================================================================
//  Program: NMEA_isr.ino
//
//  Prerequisites:
//     1) NMEA.ino works with your device
//
//  Description:  This minimal program parses the GPS data during the
//     RX character interrupt.  The ISR passes the character to
//     the GPS object for parsing.  The GPS object will add gps_fix
//     structures to a buffer that can be later read() by loop().
//======================================================================

#if defined( UBRR1H ) | defined( ID_USART0 )
// Default is to use NeoSerial1 when available.  You could also
#include <NeoHWSerial.h>
// NOTE: There is an issue with IDEs before 1.6.6.  The above include
// must be commented out for non-Mega boards, even though it is
// conditionally included.  If you are using an earlier IDEs,
// comment the above include.
#else
// Only one serial port is available, uncomment one of the following:
//#include <NeoICSerial.h>
//#include <NeoSWSerial.h>
//#include <SoftwareSerial.h> /* NOT RECOMMENDED */
#endif
#include "GPSport.h"

#include "Streamers.h"
#ifdef NeoHWSerial_h
#define DEBUG_PORT NeoSerial
#else
#define DEBUG_PORT Serial
#endif

// Check configuration

#ifndef NMEAGPS_INTERRUPT_PROCESSING
#error You must define NMEAGPS_INTERRUPT_PROCESSING in NMEAGPS_cfg.h!
#endif

static NMEAGPS   gps;

//--------------------------

static void GPSisr( uint8_t c )
{
  gps.handle( c );

} // GPSisr

//--------------------------

void deletescreen()
{
  mydisp.clearScreen();
  mydisp.setRotation(1);
  mydisp.setColor(20);
  mydisp.drawFrame(0, 0, 318, 30);
  mydisp.setColor(1);
  mydisp.setFont(18);
  mydisp.setPrintPos(1, 0, _TEXT_);
  mydisp.setTextPosOffset(0, 9);
  mydisp.print( F("TEST") );
  mydisp.setFont(0);
}

void setup()
{
  // Start the normal trace output
  DEBUG_PORT.begin(115200);
  while (!DEBUG_PORT)
    ;

  DEBUG_PORT.print( F("NMEA_isr.INO: started\n") );
  DEBUG_PORT.print( F("fix object size = ") );
  DEBUG_PORT.println( sizeof(gps.fix()) );
  DEBUG_PORT.print( F("NMEAGPS object size = ") );
  DEBUG_PORT.println( sizeof(gps) );
  DEBUG_PORT.println( F("Looking for GPS device on " USING_GPS_PORT) );

  trace_header( DEBUG_PORT );

  DEBUG_PORT.flush();

  // Start the UART for the GPS device
  gps_port.attachInterrupt( GPSisr );
  gps_port.begin( 115200 );

  pinMode(splitsim, INPUT); //simulate split
  digitalWrite(splitsim, HIGH);
  mydisp.begin();
  deletescreen();
  startlap = millis();
}

//--------------------------

void loop()
{

  if (gps.available()) {
    // Print all the things!
    trace_all( DEBUG_PORT, gps, gps.read() );
  }

  if (gps.overrun()) {
    gps.overrun( false );
    DEBUG_PORT.println( F("DATA OVERRUN: took too long to use gps.read() data!") );
  }

  if ((digitalRead(splitsim) == 0) && (split == 0)) {
    split = 1;
    if (l < MAX_LAPTIMES) {
      laptimes[l] = laptime;
      l++;
    }
    startlap = millis();
  }
  if (seconds0 == 1)
  {
    split = 0;
  }

  laptime     = millis() - startlap;
  miliseconds = laptime % 1000;
  seconds0    = (laptime % 10000) / 1000;
  seconds1    = (laptime / 10000) % 6;
  minutes     = laptime / 60000;

  if (miliseconds % 50 == 0)
  {
    mydisp.setPrintPos(0, 6, _TEXT_);
    mydisp.print( F("Lap:") );

    mydisp.setColor(31);
    mydisp.setPrintPos(5, 6, _TEXT_);
    if (minutes < 10)
      mydisp.print( '0' );
    mydisp.print(minutes);

    mydisp.setPrintPos(7, 6, _TEXT_);
    mydisp.print( ':' );
    mydisp.setPrintPos(8, 6, _TEXT_);
    mydisp.print(seconds1);
    mydisp.setPrintPos(9, 6, _TEXT_);
    mydisp.print(seconds0);
    mydisp.setPrintPos(10, 6, _TEXT_);
    mydisp.print( '.' );
    mydisp.setPrintPos(11, 6, _TEXT_);
    if (miliseconds < 100)
      mydisp.print( '0' );
    if (miliseconds < 10)
      mydisp.print( '0' );
    mydisp.print(miliseconds);
    mydisp.setColor(1);
  }
}


believe it or not, the starttime variable name was the problem. perhaps it is used inside the neogps libraries? i don't know. i noticed that if in setup i set it to =0 it worked, then i just changed the name to startlap and it is fine, starts from zero.

the above code now works great. its the NMEA_isr example and i added stuff directly in loop(). even if i change the screen refresh to %20 and scramble the screen, the serial monitor is still intact.


-dev

Quote
believe it or not, the starttime variable name was the problem.
Not.  It's not in NeoGPS, the Arduino core, nor any library I have downloaded, and I have downloaded quite a few.  There are some "startTime" variables in a few ethernet-related files.  It might be in the Digole library, but you would normally get a link error when two files have the same variable.

If you search and replace "startlap" with "starttime" in your current program, the initial value is not correct?  I'm skeptical.

Quote
the above code now works great.
Yay! 

One thing to be aware of...  The Arduino millis() clock drifts against the GPS time, so it's usually better to use the GPS time for tasks that are related to GPS information.

For example, if you are saving/sending/printing locations, don't do this:

Code: [Select]
  if (gps.available())
    fix = gps.read();

  if (millis() - lastDisplay > 1000) {
    lastDisplay += 1000;
    sendLocation();
  }

This will occasionally use the same location twice.  Other times, it will miss using a location.  How often this happen depends on the (in)accuracy of the Arduino crystal.

It is completely safe to use locations like this:

Code: [Select]
  if (gps.available()) {
    fix = gps.read();
    printLocation();
  }

Or, if you are only doing something once every 20 seconds (1 out of 20 fixes), you can do this:

Code: [Select]
  if (gps.available()) {
    fix = gps.read();
    if (++fixCount >= 20) {
      fixCount = 0;
      updateDisplay();
    }
  }

You can also use the fix.dateTime_cs:

Code: [Select]
  if (gps.available()) {
    fix = gps.read();
    if (fix.valid.time && (fix.dateTime_cs < GPS_UPDATE_PERIOD/10)) { // once per second
      saveLocation();
    }
  }

This keeps the save/send/print "in sync" with the GPS updates: none are doubled and none are lost.  Since you're just displaying the information, it's not a big deal.

It will change the processing load, though.  As the clock drifts, your tasks will sometimes be performed during the GPS quiet time (good).  Other times, your tasks will be performed while the GPS is sending information (bad).  The processor has to handle your tasks and the character interrupt/parsing at the same time.  You are using the interrupt-driven approach, so this won't have much effect.  It could slow down your tasks by about 1ms/update.  Again, it's not a big deal, just an FYI.

Cheers,
/dev

fuzion

once again, great info. learned so much from all this.

interesting timing approach, i am wondering, which timer would be better to use for my app, millis or gps.

the thing is i will be using only gps positions. i have saved the lat lon of a track finish line, and if the position i am at the moment is less than 0.0004 in decimal degrees both lat long, i reset the lap and save the last one.

i think that millis is a better choice, as i think is closer to real .050's of time comparing to messages number received. but this is also kind of overkill as gps messages are also more accurate than 50ms.

but what happens if a message is lost? i like how the neogps doesn't mess the data, it just doesnt parse it at all, but then i will miss a whole 50ms.

or should i use gps time and milliseconds to record laptimes instead of gps messages? perhaps that would be the safest.

on the variable name thing, i really don't know what the issue was. i displayed on the screen the starttime value in the loop, after declaring starttime=millis() at the setup, and i got a huge number. then inside the loop i did once starttime=0, and then starttime=millis, and got 0. this indicated that after the setup, starttime was getting some other value. when i changed the name, it was gone.

i also believe this was some kind of bug in my program. if i did it from scratch perhaps it would be gone. i dont know now.

bonus issues:

i couldnt get the fix.latitude() or fix.longitude() to display. couldnt convert them to string also. i have to do some more reading on neogps and its output, i am doing something wrong.

does neogps give distance travelled i wonder.... if not, i have to calculate it through lat lon or speed and time perhaps

-dev

Quote from: Fuzion
i am wondering, which timer would be better to use for my app, millis or gps.
Heh, heh.  It depends on what you're using the timer for...

If you are blinking an LED, just use millis().  As we know, this GPS device does not provide the fixes at an exact 50ms interval.  The fix information is spaced at 50ms intervals, but they are received at irregular intervals.  So if your timed task needs regularity, and/or it's a fairly simple task, just use millis().

If you are doing something "expensive" (e.g., printing, writing to an SD card, or reading a sensor) or something that happens when you get new GPS information (e.g., speed limit, geofence, or lap timer), coordinate it with the fix arrivals to balance the CPU load.  When a new fix becomes available, the GPS quiet time has started.  That's the best time to do the heavy lifting.

Quote
i think that millis is a better choice, as i think is closer to real .050's of time comparing to messages number received. but this is also kind of overkill as gps messages are also more accurate than 50ms.
Yes, millis() is definitely better for relative times where accuracy isn't important AND when you're doing something simple.  Conversely, the GPS times are atomically accurate absolute times, and they come in batches at irregular intervals.

Quote
i have saved the lat lon of a track finish line, and if the position i am at the moment is less than 0.0004 in decimal degrees both lat long, i reset the lap and save the last one...  but what happens if a message is lost? ... then i will miss a whole 50ms.
Yes, that is why I pointed you to this thread.  It calculates the crossing time by "intersecting" the vehicle path segment (two GPS fixes) with the finish line segment (two locations from Google maps):



When you use a "line-crossing" calculation, it doesn't matter if you lose an update.  The math uses the timestamp in the fixes to calculate the intersection time.  The code is in reply #15; here's the juicy bits:

Code: [Select]
static void Intersection()
{
    s1_x = p1_x - p0_x;                             // Finish line segment is p0-p1
    s1_y = p1_y - p0_y;
    s2_x = fix.longitudeL() - lastFix.longitudeL(); // Vehicle path segment is fix-lastFix
    s2_y = fix.latitudeL() - lastFix.latitudeL();

    s = (-s1_y * (p0_x - lastFix.longitudeL()) + s1_x * (p0_y - lastFix.latitudeL())) / (-s2_x * s1_y + s1_x * s2_y);
    t = ( s2_x * (p0_y - lastFix.latitudeL()) - s2_y * (p0_x - lastFix.longitudeL())) / (-s2_x * s1_y + s1_x * s2_y);

    // see if they intersect (t and s both between 0 and 1)
    if ((0 <= s) && (s <= 1) && (0 <= t) && (t <= 1)) {

        // Intersected!
        i_x = p0_x + (t * s1_x); //Intersection Longitude
        i_y = p0_y + (t * s1_y); //Intersection Latitude

        dt = lastFix.dateTime_cs - fix.dateTime_cs;
        if (dt < 0)
          dt += 100;
        i_t  = s * dt;   // Intersection Time as 0.01s offset from lastFix.
   }

   lastFix = fix; // save it for next time

} // Intersection

BTW, that sketch uses the older character-oriented methods (decode).  You should stick with the fix-oriented methods (available and read).

Quote
i couldnt get the fix.latitude() or fix.longitude() to display.
Show.  Code.  :)  Or look at NMEAsimple.ino.  Maybe the Digole library does not support floating-point output?

Quote
couldnt convert them to string
I'm sure you mean a C string, aka char arrays, 'cuz nobody should use the String class.  There is a way, but why do you need a string?  Everything you've mentioned so far could use a print method.  It's always better to print the pieces than to put all the pieces in one big string, and then print the big string.

Quote
does neogps give distance travelled? if not, i have to calculate it through lat lon
No, and correct.  Like this.

Code: [Select]
float odo; // km

void loop()
{
  if (gps.available()) {
    fix = gps.read();

    if (fix.valid.location && lastGoodFix.valid.location) {
      odo += fix.location.DistanceKm( lastGoodFix.location );
      lastGoodFix = fix;  // save for next time
    }

You may want to skip this when the speed < 5kph.  GPS "wanders" all the time, so if you're stationary you'd be adding up the jitter, accumulating distance.  :P

Cheers,
/dev

fuzion

as always, eye-opening info.

will post after all this is tested thanks.

Go Up