Go Down

Topic: GPS data analysis (Read 788 times) previous topic - next topic

OhMyCod

Hi,

I'm trying to analyse how often (and how much) data a GPS module is generating. I started with this very simple loop.....
Code: [Select]
    for (int i=0 ;i<36000; i++)
    {
      bytesRead = 0;
      while (Serial1.available())
      {
        // Read (and ignore) a SINGLE byte from the port
        Serial1.read();
        bytesRead++;
      }
      Serial.print("Read <");Serial.print(bytesRead);Serial.print("> bytes from GPS port at <");Serial.print(millis());Serial.println("> milli seconds since startup.");
      bytesRead = 0;
    }


this gives no problems and gives me a result like this.....
Quote
Read <0> bytes from GPS port at <1433> milli seconds since startup.
Read <0> bytes from GPS port at <1438> milli seconds since startup.
Read <0> bytes from GPS port at <1444> milli seconds since startup.
Read <35> bytes from GPS port at <1451> milli seconds since startup.
Read <34> bytes from GPS port at <1457> milli seconds since startup.
Read <34> bytes from GPS port at <1463> milli seconds since startup.
Read <33> bytes from GPS port at <1468> milli seconds since startup.
Read <34> bytes from GPS port at <1474> milli seconds since startup.
Read <34> bytes from GPS port at <1480> milli seconds since startup.
Read <33> bytes from GPS port at <1486> milli seconds since startup.
Read <34> bytes from GPS port at <1492> milli seconds since startup.
Read <33> bytes from GPS port at <1498> milli seconds since startup.
Read <34> bytes from GPS port at <1504> milli seconds since startup.
Read <34> bytes from GPS port at <1510> milli seconds since startup.
Read <33> bytes from GPS port at <1516> milli seconds since startup.
Read <11> bytes from GPS port at <1522> milli seconds since startup.
Read <0> bytes from GPS port at <1527> milli seconds since startup.
Read <0> bytes from GPS port at <1533> milli seconds since startup.
Read <0> bytes from GPS port at <1540> milli seconds since startup.


Which is exactly what was expected - it tells me at what moment the GPS sent a burst of data.

I then decided to modify the routine to totalise the amount of data received in each burst. this is what I wrote....
Code: [Select]
    for (int i=0 ;i<36000; i++)
    {
      while (Serial1.available())
      {
        // Read (and ignore) a SINGLE byte from the port
        Serial1.read();
        bytesRead++;
      }
     
      // totalise the amount of data received in each 'burst'
      if ( bytesRead != 0 )
      {
        totBytesRead = totBytesRead + bytesRead;
        burst++;
      }
     
      Serial.print("bytesRead=<");Serial.print(bytesRead);Serial.print(">, totBytesRead=<");Serial.print(totBytesRead);Serial.print(">, burst=<");Serial.print(burst);Serial.print("> prevBytesRead=<");Serial.print(prevBytesRead);Serial.println(">");
     
      if( (bytesRead == 0) && (prevBytesRead!=0) )
      {
        Serial.print("Read <");Serial.print(totBytesRead);Serial.print(">bytes, in <");Serial.print(burst);Serial.print(">bursts, from port at <");Serial.print(millis());Serial.println("> milli secs since start.");
        totBytesRead=0;
        burst=0;
      }
     
      prevBytesRead = bytesRead;
      bytesRead = 0;
    }


As before, this produced exactly what I expected.....
Quote
bytesRead=<0>, totBytesRead=<0>, burst=<0> prevBytesRead=<0>
bytesRead=<0>, totBytesRead=<0>, burst=<0> prevBytesRead=<0>
bytesRead=<0>, totBytesRead=<0>, burst=<0> prevBytesRead=<0>
bytesRead=<3>, totBytesRead=<3>, burst=<1> prevBytesRead=<0>
bytesRead=<32>, totBytesRead=<35>, burst=<2> prevBytesRead=<3>
bytesRead=<30>, totBytesRead=<65>, burst=<3> prevBytesRead=<32>
bytesRead=<32>, totBytesRead=<97>, burst=<4> prevBytesRead=<30>
bytesRead=<31>, totBytesRead=<128>, burst=<5> prevBytesRead=<32>
bytesRead=<32>, totBytesRead=<160>, burst=<6> prevBytesRead=<31>
bytesRead=<31>, totBytesRead=<191>, burst=<7> prevBytesRead=<32>
bytesRead=<32>, totBytesRead=<223>, burst=<8> prevBytesRead=<31>
bytesRead=<32>, totBytesRead=<255>, burst=<9> prevBytesRead=<32>
bytesRead=<31>, totBytesRead=<286>, burst=<10> prevBytesRead=<32>
bytesRead=<33>, totBytesRead=<319>, burst=<11> prevBytesRead=<31>
bytesRead=<32>, totBytesRead=<351>, burst=<12> prevBytesRead=<33>
bytesRead=<32>, totBytesRead=<383>, burst=<13> prevBytesRead=<32>
bytesRead=<32>, totBytesRead=<415>, burst=<14> prevBytesRead=<32>
bytesRead=<6>, totBytesRead=<421>, burst=<15> prevBytesRead=<32>
bytesRead=<0>, totBytesRead=<421>, burst=<15> prevBytesRead=<6>
Read <421>bytes, in <15>bursts, from port at <3142> milli secs since start.


Ok, now I'm getting to the hub of the matter. The only part of the output I'm actually interested in is the line...
Quote
Read <421>bytes, in <15>bursts, from port at <3142> milli secs since start


So, all I need to do is comment out this line...
Code: [Select]
      Serial.print("bytesRead=<");Serial.print(bytesRead);Serial.print(">, totBytesRead=<");Serial.print(totBytesRead);Serial.print(">, burst=<");Serial.print(burst);Serial.print("> prevBytesRead=<");Serial.print(prevBytesRead);Serial.println(">");


However, as soon as I remove this line (which does nothing but display to the screen) the results actually get changed.
This is what I get.....
Quote
Read <1>bytes, in <1>bursts, from port at <1868> milli secs since start.
Read <18>bytes, in <2>bursts, from port at <1871> milli secs since start.
Read <24>bytes, in <1>bursts, from port at <1878> milli secs since start.
Read <37>bytes, in <1>bursts, from port at <1884> milli secs since start.
Read <35>bytes, in <1>bursts, from port at <1890> milli secs since start.
Read <37>bytes, in <1>bursts, from port at <1896> milli secs since start.
Read <36>bytes, in <1>bursts, from port at <1903> milli secs since start.
Read <36>bytes, in <1>bursts, from port at <1909> milli secs since start.
Read <36>bytes, in <1>bursts, from port at <1915> milli secs since start.
Read <36>bytes, in <1>bursts, from port at <1922> milli secs since start.
Read <36>bytes, in <1>bursts, from port at <1929> milli secs since start.
Read <36>bytes, in <1>bursts, from port at <1935> milli secs since start.
Read <36>bytes, in <1>bursts, from port at <1941> milli secs since start.
Read <36>bytes, in <1>bursts, from port at <1947> milli secs since start.
Read <15>bytes, in <1>bursts, from port at <1953> milli secs since start.
Read <1>bytes, in <1>bursts, from port at <2068> milli secs since start.
Read <18>bytes, in <1>bursts, from port at <2071> milli secs since start.


Compared to what I expected....
Quote
Read <470>bytes, in <16>bursts, from port at <211> milli secs since start.
Read <470>bytes, in <15>bursts, from port at <408> milli secs since start.
Read <470>bytes, in <16>bursts, from port at <610> milli secs since start.
Read <470>bytes, in <16>bursts, from port at <812> milli secs since start.
Read <470>bytes, in <16>bursts, from port at <1008> milli secs since start.
Read <470>bytes, in <16>bursts, from port at <1210> milli secs since start.
Read <470>bytes, in <16>bursts, from port at <1413> milli secs since start.
Read <470>bytes, in <16>bursts, from port at <1609> milli secs since start.
Read <470>bytes, in <16>bursts, from port at <1812> milli secs since start.
Read <470>bytes, in <16>bursts, from port at <2009> milli secs since start.


On the plus side, I've actually got the results I was after by hand-editing the screen output, but on the downside there's obviously something going wrong that I've missed. How can the inclusion of a series of print() statements effect the actual results? and how can I cure it.

Thanks

mrburnette

Look over what a GPS typically sends to the host:
http://www.maartenlamers.com/nmea/

Your character streams are non-blocked and are uniquely formatted.  Generally, we in the Arduino community use a GPS library to parse what is needed.  Some write their own routines.  Check out the standard libraries: http://playground.arduino.cc//Main/LibraryList
or use Google to get a larger list.


Ray

Frédéric_Plante

I'm gonna go tru you lines of code.

But One thing is for sure. Programming every thing on a single line look good in you code. But it make it harder to debug when there is a problem. May I suggest to do a multi-line until everything work?
As we fight our way northward into the great unknown, only that one thing remains certain...

OhMyCod

Thanks mrburnette

I know what a GPS typically sends, and I do have a functioning GPS library. I've been investigating how often (and how many) characters are been generated by the GPS - at this moment I'm not actually interested in what the data is.

The problem is this - the code shown above works fine when I include a set of print() statements, but as soon as I comment out that line, the actual results change!

With the print() line enabled I see that I get a block of about 470 characters delivered in 16 'bursts' of data - but as soon as I remove the print() line it changes to show me getting about 30-40 bytes delivered in a single block. Why? the print() statements shouldn't change anything

Cheers

Frédéric_Plante

Ok, let's see
Code: [Select]

Serial.print("bytesRead=<");
Serial.print(bytesRead);
Serial.print(">, totBytesRead=<");
Serial.print(totBytesRead);
Serial.print(">, burst=<");
Serial.print(burst);
Serial.print("> prevBytesRead=<");
Serial.print(prevBytesRead);
Serial.println(">");
As we fight our way northward into the great unknown, only that one thing remains certain...

OhMyCod

Quote
I'm gonna go tru you lines of code.

Thanks

I only multi-line the print() statements - didn't see much harm in it myself.

cheers

Frédéric_Plante

Yeah that's what i'm seeing to. ;)

But, you can't catch fly with vinegar, so if some one see that he is gonna have to scroll left and right to understand your stuff, he might give up before even noticing that it's not the important part.

You know that it's not important, but we don't, until we read it. And at first look, it could scare the less courageous one that could still find the answer right away if you sell your stuff right.
As we fight our way northward into the great unknown, only that one thing remains certain...

el_supremo

The lines you are printing are quite long. The serial output buffer is (IIRC) 32 bytes, so the output will block until there's less than 32 bytes in the buffer. Changing the amount of data printed changes the timing of when, how often and for how long it blocks.
Simplest thing is to change the serial output speed (and the serial monitor) to 57600 or even higher.

Pete

OhMyCod

The serial monitor is already set to 115200;

I've got to admit that I've never had a problem because I'm printing out too much!

thanls

el_supremo

Change the output to print only the two numbers separated by a space. See if that changes and/or helps things.

Pete

1ChicagoDave




Code: [Select]

      if( (bytesRead == 0) && (prevBytesRead!=0) )
      {
        Serial.print("Read <");Serial.print(totBytesRead);Serial.print(">bytes, in <");Serial.print(burst);Serial.print(">bursts, from port at <");Serial.print(millis());Serial.println("> milli secs since start.");
        totBytesRead=0;
        burst=0;
      }
     
      prevBytesRead = bytesRead;
      bytesRead = 0;
    }



I may be way off base here (and not at all properly explaining what I'm seeing), but....

Two things --

Looks like you're resetting totBytesRead, burst, & bytesRead to 0 after printing the values, no?

Without the extra print statements (to slow things down) could the loop be running through again before more data is sent/received by GPS?

OhMyCod

Quote
Without the extra print statements (to slow things down) could the loop be running through again before more data is sent/received by GPS?


This is correct. The problem was actually solved in about 10 seconds flat by my wife, if I take out the series of print() statements and insert a delay of 5 milliseconds then I get exactly the same results. I knew a print() would be time consuming, but I had no idea it would take so long. Basically the delay was allowing more time for the incoming data to be queued up at the input port for processing.

well, I know who to go to first next time I have a problem.

Thanks for everyone's contributions.

1ChicagoDave


Quote
Without the extra print statements (to slow things down) could the loop be running through again before more data is sent/received by GPS?


This is correct. The problem was actually solved in about 10 seconds flat by my wife, if I take out the series of print() statements and insert a delay of 5 milliseconds then I get exactly the same results. I knew a print() would be time consuming, but I had no idea it would take so long. Basically the delay was allowing more time for the incoming data to be queued up at the input port for processing.

well, I know who to go to first next time I have a problem.

Thanks for everyone's contributions.


Sweet!

Yep....the wife always knows.   :smiley-roll:

Nick Gammon


This is correct. The problem was actually solved in about 10 seconds flat by my wife, if I take out the series of print() statements and insert a delay of 5 milliseconds then I get exactly the same results. I knew a print() would be time consuming, but I had no idea it would take so long.


Adding a delay "solves" it but the wrong way. Read this:

http://www.gammon.com.au/serial

This is wrong, for example:

Quote

Code: [Select]

while (Serial1.available())
      {
        // Read (and ignore) a SINGLE byte from the port
        Serial1.read();
        bytesRead++;
      }
      Serial.print("Read <");Serial.print(bytesRead);Serial.print("> bytes from GPS port at <");Serial.print(millis());Serial.println("> milli seconds since startup.");



You are printing even if there is nothing available. Clearly the print should be inside the "while available" not outside.
Please post technical questions on the forum, not by personal message. Thanks!

More info:
http://www.gammon.com.au/electronics

Nick Gammon

Even that isn't quite right because it would print a line for every received byte.

Read the link about doing serial reads, loops with delays usually are the wrong way of doing it.
Please post technical questions on the forum, not by personal message. Thanks!

More info:
http://www.gammon.com.au/electronics

Go Up