TWI losing data

Using Arduino Networking, Protocols, and Devices
1

I’ve got a project that hooks up a 2004 LCD display & 10Hz GPS - both via TWI. The microcontroller is 5V, as is the LCD. The GPS is 3V3, so I am hooking that up via a level converter break out board, which incorporates 10K pull up resistors on both sides. The LCD seems to be rock solid, but I am having problems with reliability of communication between microcontroller and gps. Generally I am able to configure the GPS to report 10Hz and just deliver one sentence (<80 characters).

Once set up the messages validate, but sometimes, and this can happen several times per second, the validity checks fail and by observation the failure is loss of a single character. I reckon the occasional failure to configure the gps is down to much the same problem - the gps doesn’t get the same sentence that the microcontroller wrote.

Both lcd & gps are within about 150mm of microcontroller (Arduino Nano or Nano Every). Symptoms don’t seem noticeably different whether running the TWI bus at normal clock rate (100khz) or fast (400khz?)

I’m really putting this out to enquire whether any of you have come across this problem. I have tried powering this through the usb host and from pp3 battery to Vin - no difference. I have tried removing the lcd to exclude it both as interference & to ensure it’s not making excessive demands on the on board (microcontroller) voltage regulator.

The character outage does seem to be random but unreasonably frequent.

Any anecdotal experience or suggestions would be appreciated.

2

On some boards, the buffer for the I2C bus is 32 bytes. NMEA sentences from the GPS module often exceed that so these must be broken down, possible to single characters. Note also that most libraries for the I2C LCD screens block for some milliseconds on some operations like clear screen which could be enough time to lose a character or so elsewhere.
Maybe start by saying what hardware you are using and showing your code.
Also try without the LCD code and simply write to the serial console at a high baud rate.

3

@6v6gt - great message. That prompted me to extract the salient code into a separate sketch.

#include "SparkFun_I2C_GPS_Arduino_Library.h"

const byte bMaxSentence = 85; // NMEA specifies 82 max
const long MONITOR_BAUDRATE = 250000;

I2CGPS myI2CGPS; //Hook object to the library
char NMEAsentence[bMaxSentence];

void setup() 
{
  String strConfigString;

  Serial.begin(MONITOR_BAUDRATE);
  // I2C_SPEED_STANDARD = 100000, I2C_SPEED_FAST = 400000
  if (!myI2CGPS.begin(Wire, I2C_SPEED_STANDARD)) 
  {
    Serial.println("Module failed to respond. Please check wiring.");
    for(;;);  // Hang forever
  }

  Serial.println("GPS module found!");

  strConfigString = myI2CGPS.createMTKpacket(220, ",1000"); // 100 for 10Hz, 1000 for 1Hz
  myI2CGPS.sendMTKpacket(strConfigString); // Set Update Rate
  strConfigString = myI2CGPS.createMTKpacket(314, ",0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0");
  myI2CGPS.sendMTKpacket(strConfigString); // Just enable $GNRMC messages
}

void loop() 
{
  byte bNMEAindex = NULL;
  bool bDollarFound = false;

  while(myI2CGPS.available())  // Discard until $
  {
    if (myI2CGPS.read() == '$')
    {
      bDollarFound = true;  // Flag success for next stage
      NMEAsentence[bNMEAindex++] = '$';
      break;
    }
  }

  if(bDollarFound)
  {
    while((myI2CGPS.available()) && (bNMEAindex < (bMaxSentence - 4)))
    {
        NMEAsentence[bNMEAindex] = myI2CGPS.read();
        if (NMEAsentence[bNMEAindex++] == '*')
        {
          NMEAsentence[bNMEAindex++] = myI2CGPS.read();
          NMEAsentence[bNMEAindex++] = myI2CGPS.read();  // Capture the checksum
          break;
        }
    }
    NMEAsentence[bNMEAindex] = NULL;

    if(bNMEAindex != 74) // I know a well formed $GNRMC valid sentence is 74
    {
      Serial.println(bNMEAindex);
    };      
    Serial.println(NMEAsentence);
  }
}

I bought the GPS from Cool Components. They call it their GPS Breakout - XA1110. They also give what I guess to be their part number - GPS-14414. In their support documentation they provide a datasheet for Titan X1 GPS. I am satisfied that the buffer GPS side of the TWI is much bigger than 32 bytes. I think I have seen it's 1k or .5k, but can't lay hands on my reference. Anyway, I am satisfied it's big enough to hold several sentences because I can pull quite a few if it has been powered up before I start. The upside of that is I can ignore the GPS for a second or so, & catch up with what's in its buffer in my own time. In any event, my test program isn't talking to the LCD, so that's not the problem. Test program runs monitor at 250 kbps. Same symptoms whether that or 9600.

Here's an extract of my monitor output.

3:21:12.551 -> $GNRMC,122113.000,A,5051.176925,N,00100.473457,W,1.30,123.78,210823,,,A*6F
13:21:13.576 -> 73
13:21:13.576 -> $GNRMC,122114.000,A,5051.176913,N,00100.473920,W,0.79,13.78,210823,,,A*6C
13:21:14.572 -> 73
13:21:14.572 -> $GNRMC122115.000,A,5051.176926,N,00100.474218,W,0.59,123.78,210823,,,A*6E
13:21:15.567 -> $GNRMC,122116.000,A,5051.177098,N,00100.474691,W,0.18,123.78,210823,,,A*60
13:21:16.563 -> $GNRMC,122117.000,A,5051.177087,N,00100.475307,W,0.26,123.78,210823,,,A*69
13:21:17.548 -> $GNRMC,122118.000,A,5051.177106,N,00100.475942,W,0.29,123.78,210823,,,A*6A
13:21:18.588 -> $GNRMC,122119.000,A,5051.176843,N,00100.476257,W,0.59,123.78,210823,,,A*69
13:21:19.573 -> 73
13:21:19.573 -> $GNRMC,122120.000,A,5051.176506N,00100.476928,W,0.48,123.78,210823,,,A*6C
13:21:20.556 -> $GNRMC,122121.000,A,5051.176677,N,00100.476227,W,0.48,123.78,210823,,,A*6C
13:21:21.531 -> $GNRMC,122122.000,A,5051.176544,N,00100.476028,W,0.90,123.78,210823,,,A*64
13:21:22.543 -> $GNRMC,122123.000,A,5051.176521,N,00100.475911,W,0.37,123.78,210823,,,A*6B
13:21:23.589 -> $GNRMC,122124.000,A,5051.176673,N,00100.476127,W,0.16,123.78,210823,,,A*65
13:21:24.553 -> $GNRMC,122125.000,A,5051.176633,N,00100.476105,W,0.39,123.78,210823,,,A*6D
13:21:25.570 -> $GNRMC,122126.000,A,5051.176743,N,00100.476152,W,0.42,123.78,210823,,,A*66
13:21:26.601 -> 73
13:21:26.601 -> $GNRMC,122127.000,A,5051.176643,N,00100.476225,W,0.38,123.78,210823,,A*68
13:21:27.577 -> 73
13:21:27.577 -> $GNRMC,122128.000,A,501.176669,N,00100.476427,W,0.19,123.78,210823,,,A*68
13:21:28.564 -> $GNRMC,122129.000,A,5051.176634,N,00100.476827,W,0.45,123.78,210823,,,A*64
13:21:29.584 -> $GNRMC,122130.000,A,5051.176627,N,00100.477077,W,0.58,123.78,210823,,,A*6E
13:21:30.517 -> $GNRMC,122131.000,A,5051.176477,N,00100.477491,W,0.61,123.78,210823,,,A*6E
13:21:31.540 -> $GNRMC,122132.000,A,5051.176406,N,00100.477711,W,0.59,123.78,210823,,,A*6B
13:21:32.573 -> $GNRMC,122133.000,A,5051.176482,N,00100.477886,W,0.38,123.78,210823,,,A*60
13:21:33.559 -> $GNRMC,122134.000,A,5051.176566,N,00100.478115,W,0.36,123.78,210823,,,A*6E
13:21:34.587 -> $GNRMC,122135.000,A,5051.176860,N,00100.478744,W,0.07,123.78,210823,,,A*64
13:21:35.559 -> $GNRMC,122136.000,A,5051.177194,N,00100.479717,W,0.81,123.78,210823,,,A*6D
13:21:36.535 -> $GNRMC,122137.000,A,5051.177158,N,00100.480739,W,0.87,293.00,210823,,,A*67
13:21:37.580 -> 73
13:21:37.580 -> $GNRMC,122138.000,A,5051.177364,N,00100.481151,W,0.61,336.16,21823,,,A*6D
13:21:38.583 -> $GNRMC,122139.000,A,5051.177534,N,00100.481876,W,0.82,304.34,210823,,,A*6F
13:21:39.564 -> $GNRMC,122140.000,A,5051.177736,N,00100.482088,W,0.41,304.34,210823,,,A*64
4

Your code looks OK although there are a couple of places where you are doing a read() without checking available(), however, that would not give the missing character problem your output shows. More likely, you would then get spurious 0 characters if no character was available in the buffer.

I'd probably start with your setup() to restrict the output to $GNRMC messages but then take the simpler loop() from here: https://github.com/sparkfun/SparkFun_I2C_GPS_Arduino_Library/blob/master/examples/Example1-BasicReadings/Example1-BasicReadings.ino to see if the missing character problem still persists.

I guess, though, it is going to be a problem between the library and the GPS module and the simplest solution / work-around is to apply the check sum test and discard invalid packets.

5

Whilst you are correct that I don't check for available when collecting the checksum characters, the run time evidence tells us we're not gaining anything that the gps hasn't sent. Actually, I have been satisfied that the GPS does dump complete sentences into its outbound buffer. Had that not been so, the output log would have shown truncated sentences (missing stuff on the right side of the line). That's not happening. From pattern matching we can see the missing characters are at fairly random positions from the sentence.

I would actually argue that the variation you propose would not contribute to any improvement that I can see. You won't be surprised that my test program was extracted from other code where I am collecting the sentence for later processing. There's a danger in introducing a variation that, to me, at least, does not deliver a demonstrable and significant benefit. I think it diverts attention from the actual problem at hand. To what extent is this simply a matter of "style"?

I don't have a problem detecting the duff sentences & ignoring them. The problem I raised is that the frequency of their occurrence is much higher than I would expect. I hope you would agree that the output log is adequate to expose the problem. Arguably, more rigorous validation can be carried out. Statistically I don't think the solution / work around you propose would make a significant difference to the number of false passes or false fails - but in the end, I was only to put out a fairly minimalist program to demonstrate the issue.

Your input has been most helpful. Coming from an historic background of having a physical peer group to bounce ideas off, to a fairly isolated environment, it's a real joy to have this productive exchange of thoughts. Thank you. ... and ... you did prompt me into separating out the problem into a small self contained program to illustrate the issue.

6

I don’t know what sort of handshaking is going on between the microcontroller and the gps across the TWI, but it is worthy of note that the only apparent manifestation of the fault is the occasional loss of a single character. Never the inversion of any bit(s) within a character. Never the pick up of a spurious extra character. This might be because there are some extra check bits at the character level, but if that’s so, what’s the reasoning for doing that silently - leaving no witness to the event. This kind of steers my thinking away from electrical level issues. Noise on the wiring would deliver bit level errors, I think. Not the complete omission of whole characters.

In parallel with me trying to bottom out this problem I have ordered another GPS from cool components (Adafruit?). It’s Adafruit Mini GPS PA1010D - UART and I2C - STEMMA QT.

This is described as “5V friendly” so I am hoping I can connect it directly to a 5V Arduino. Untypically, I have only purchased a single GPS receiver. I do like to have more than one sample to play with, but not this time.

@6v6gt - you made mention of a 32 character buffer in your first reply. I presumed you were talking about the gps side of the TWI, but could it be down to buffering on the Arduino side? I guess I can answer my own question. I understand the “available” function actually tells the caller how many characters can be pulled. I am expecting that to be ~74 when I start collecting the sentence. Thinking out loud, really. I’m on my ‘phone & my Windows PC is out of reach. A little task for tomorrow.

7

Well, I have made some progress - but not in a good way!

I have bought the Adafruit Mini GPS PA1010D. It also uses the MT3333 GPS engine, and provides TWI connectivity, but offers the significant benefit that it's sold as being "5V friendly". I have put that onto a prototype breadboard with just a 4 wire connection, and those wires are only ~100mm. I have modified the test program to validate by checksum comparison. Bottom line - no difference in behaviour the I can spot.

Connections ground, +5 volts, TWI data (A4) and TWI clock (A5).

Symptoms seem to be loss of a single character about once every 1000 characters shipped. Data isn't corrupted. It's just whole single charaters lost. I'm getting to wonder if it's some sort of timer interrupt servicing routine that's interfering. From the code, it's clear that it's not my code if that is the case, because there would be evidence in the test program.

Then again - if some sort of timer is interfering, elevating the TWI clock from 100KHz to 400KHz should reduce the incidence of an outage.

Here's my revised code:-

#include "SparkFun_I2C_GPS_Arduino_Library.h"

const byte bMaxSentence = 85; // NMEA specifies 82 max
const long MONITOR_BAUDRATE = 250000;

I2CGPS myI2CGPS; //Hook object to the library
char NMEAsentence[bMaxSentence];

void setup() 
{
  String strConfigString;

  Serial.begin(MONITOR_BAUDRATE);
  // I2C_SPEED_STANDARD = 100000, I2C_SPEED_FAST = 400000
  if (!myI2CGPS.begin(Wire, I2C_SPEED_STANDARD)) 
  {
    Serial.println("Module failed to respond. Please check wiring.");
    for(;;);  // Hang forever
  }

  Serial.println("GPS module found!");

  strConfigString = myI2CGPS.createMTKpacket(220, ",1000"); // 100 for 10Hz, 1000 for 1Hz
  myI2CGPS.sendMTKpacket(strConfigString); // Set Update Rate
  strConfigString = myI2CGPS.createMTKpacket(314, ",0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0");
  myI2CGPS.sendMTKpacket(strConfigString); // Just enable $GNRMC messages
}

void loop() 
{
  bool bReply = false;
  byte bNMEAindex = NULL;
  bool bDollarFound = false;

  while(myI2CGPS.available())  // Discard until $
  {
    if (myI2CGPS.read() == '$')
    {
      bDollarFound = true;  // Flag success for next stage
      NMEAsentence[bNMEAindex++] = '$';
      break;
    }
  }

  if(bDollarFound)
  {
    while((myI2CGPS.available()) && (bNMEAindex < (bMaxSentence - 4)))
    {
        NMEAsentence[bNMEAindex] = myI2CGPS.read();
        if (NMEAsentence[bNMEAindex++] == '*')
        {
          NMEAsentence[bNMEAindex++] = myI2CGPS.read();
          NMEAsentence[bNMEAindex++] = myI2CGPS.read();  // Capture the checksum
          break;
        }
    }
    NMEAsentence[bNMEAindex] = NULL;

    Serial.println(NMEAsentence);
    bReply = CheckSentence(NMEAsentence);
  }
}

bool CheckSentence(char OneLine[])
{
  bool bReply = false;
  byte bChecksumCalc = 0;
  size_t  nLength = strlen(OneLine);
  const char  HexList[] = "0123456789ABCDEF";

  if ((nLength >= 4) && (OneLine[0] == '$') && (OneLine[nLength - 3] == '*') && (nLength <= bMaxSentence))
  {
    byte bColumn;

    for(bColumn = 1; bColumn < (nLength - 3); bColumn++)
    {
      bChecksumCalc ^= OneLine[bColumn];
    }

    if((HexList[bChecksumCalc >> 4]) == (OneLine[nLength - 2]))
    {
      if ((HexList[bChecksumCalc & 0x0F]) == (OneLine[nLength - 1]))
      {
        bReply = true;
      }
    }
  }

  if(!bReply)
  {
    Serial.print("Error ");
    Serial.print(HexList[bChecksumCalc >> 4]);
    Serial.println(HexList[bChecksumCalc & 0x0F]);
  }
  return bReply;
}

Ultimately I suppose I'm really hoping to connect with someone who has real experience using an MT3333 GPS engine connected to an Arduino via TWI and either has had no problem, or is aware of the problem & just lives with it.

8

OK. I can't help with this particular module. The GPS devices I have used have used a standard serial connection, that is not I2C. These are easy enough to use with SoftwareSerial.
One thing you could try is to lower the printing baud rate to see if the error rate changes proportionally but I don't hold out much hope.

9

@6v6gt - I have tried varying the printing baud rate. I have tried varying the TWI clock rate. I have tried varying the GPS sample period. Nothing seems to affect the probability of dropping a character. All I'm trying to do is ship one $GNRMC sentence every 100msec. That's less than 9600 bps.

Both of the units that I have bought offer asynchronous serial as an alternative Arduino/GPS communication path.

Somehow I can't quite believe I'm the first to exercise the TWI option ... and it really is flaky. Mayne the pragmatic approach should be to simply give up on the problem as stated. Am I just the latest of a long line of people to do so? If so, how come they have left no trace of their endeavours?

The key issue of this thread is reliability of communication between an Arduino and an MT3333 GPS via TWI. If there really is a problem, is it an MT3333 design fault or is it software in the Arduino? If it's an MT3333 problem, whether it's an Arduino on the other end wouldn't matter. Any old host would do.

@6v6gt - do you have any experience of reliable communication between an MT3333 GPS and an Arduino using asynchronous serial communication? That's a really important point. Yes, I can change to use serial comms, but what chance the same problem will manifest because the culprit is the MT3333?

10

Since you have an Adafruit Mini GPS PA1010D Module, I'd try to use the Adafruit library. See here for the details: Arduino UART Usage | Adafruit Mini GPS PA1010D Module | Adafruit Learning System for using it over Serial. If that fails in the same way you can open an incident report in the Library Github repository. I have experience only with Ublox and clones.
Maybe others here have more specific knowledge of your setup.

1 Like
11

Progress Report - I bought both GPS modules from Cool Components. A UK reseller of Sparkfun (suppliers of the first GPS module " GPS Breakout - XA1110 (Qwiic) (GPS-14414)"), and of Adafruit (suppliers of the second GPS module "Mini GPS PA1010D - UART and I2C - STEMMA QT").

To be honest, I could not have wanted more from their response. They reverted to be within about 30 minutes asking for more details. I have sent that and await their response. Very impressed.

12

Further update. Cools Components have engaged with the problem. They have been able to reproduce it themselves & have referred it to Sparkfun. Awaiting their response.

Bumbling around, not really knowing what I'm doing, I have seen a reference on the interweb to the fact that the Wire library (for TWI) has an inbound buffer of 32 characters. My understanding of TWI is that data flow between controller (the Arduino) and target (the GPS) is fundamentally under the control of the controller. Asynchronous serial communication is different. Data flow might be limited by hardware control lines (DTR/DSR) or by software signalling (XON/XOFF), but it's common for GPS equipment just to send the NMEA sentences at a rate that's simply limited by the agreed data rate. What if the wire library doesn't limit data flow from the target? What if inbound data relies on an interrupt servicing routine to trigger collection of the inbound data, which deposits it in Arduino buffer memory, and doesn't stop asking for the data even if its own buffer fills up? I have seen reference to the fact that the size of this buffer can be changed by modifying a #define in the library header Wire.h. Seems odd behaviour to me, but I am leaning to the possibility that the Wire library could be the culprit. I can relate to the possibility that data flow is delegated to an interrupt servicing routine enabling data transfer in the background - although I struggle to come to terms with a protocol that doesn't cope with blocking the inbound data stream when the controller has nowhere to put what's arrived.

13

I tend to agree here that the architecture, that is having a producer system (gps module) as a (I2C) slave of the consumer (ultimately your program), is somewhat prone to error. The gps module will have its own buffer of X bytes. The library has its own circular buffer of 256 bytes and this 256 byte buffer is refreshed periodically in chunks of max 32 bytes over I2C from the buffer in the gps module. So actually it is effectively two cascaded queues. Of course it is not impossible to get it right but it is also easy to see how it could go wrong with symptoms such as missing data, as you are seeing, or duplicated data etc.

It is good that your supplier appears to take this issue seriously. It would have been different if you had bought the modules on say Aliexpress. However, the price would also have been different. I paid about $3 dollars for the ones I bought.

14

:slightly_smiling_face: @6v6gt. I am coming to the conclusion that you and I are very much on the same wavelength, although you definitely have superior knowledge of the Arduino code.

This two level buffer management by the wire library is new to me. The thought has crossed my mind that I should be able to find the source code of the wire library to figure out how it’s behaving. I thought I could navigate from the library manager of my IDE to the source code, but haven’t cracked that one yet.

You might be interested in my last missive to Cool Components. Whilst my guesswork on the wire library does sound like it’s wrong in detail, I do believe my thinking is homing in on the same zone as you. My knowledge of the wire library is thus far purely based on my observation of its behaviour as a “black box”. I will feel a lot more comfortable once I can get the source code out on the table.

Here’s what I wrote to Cool Components:-


An idea is forming in my mind that I think I should share with you even though my knowledge of what goes on under the hood of the software at both ends is limited, verging on the non-existent.
 
TWI or I2C is a single controller / multiple target communications standard.  GPS is a target.  Arduino is the controller.  Fundamentally, the target can’t simply send data unsolicited – the controller has to ask for it.  Rummaging around on the internet, I bumped into a reference to the fact that the Arduino Wire library, which is a wrapper for managing the I2C / TWI bus, has a 32 byte input buffer. Since the controller has to ask the target for data, one would expect that the controller should know when its buffer is full, so shouldn’t ask for the next character if its buffer is already full. But, what if there’s a bug in the library software – in particular, in the incorrect management of handling the end conditions for a circular buffer (I have seen that before)?  I find it credible that there’s a bug in the Wire library.  Mostly all is well, the buffer becomes full at a position other than the end position, the Arduino knows, and doesn’t ask for another character.  But occasionally the buffer full condition coincides with the point at which buffer wrap happens.  Somehow the code is wrong, The Arduino asks the GPS for another character although it has nowhere to put it.  All might be well is the applications code calls for another character before the next one arrives from the GPS.  But if not, the character arrives from the GPS, the buffer is already full, and the handler has nowhere to put it, so it’s lost.
 
I’m getting to feel this is not a problem with my code.  It’s not a problem with the GPS.  It is a bug in the Wire library.  It does beg the question, if I am correct, why hasn’t it been spotted before? 
 
I have added a slight modification to my test program, which I attach. The key difference is it builds a metric of the proportion of failed (ie lost) messages.  For my test, it has been running at rather more than 25% of messages lost.  Sparkfun tech support might benefit from having a copy of this. (and this email!)
 
If I am correct, it could be that a GPS is a rather special example of an I2C/TWI target.  Once set going it’s trying to send a lot of data.  And it keeps doing so.  Most I2C targets tend to be quite passive – they are asked to do something, they do it, they report back.  Then they sit idle until asked again.
 
However, I think you will agree, a failure rate of 1 message in 4 is unreasonably high.  As things stand, my money is on a long standing bug in the Arduino Wire library.
15

Incidentally @6v6gt, I do agree with your assessment of my gps module source. My prospects of engaging with Alibaba on this matter would not be good.

Your description of the two level buffer architecture within the wire library is most illuminating.

16

I see that my description is a bit ambiguous because when I said "the Library", I meant this library "SparkFun_I2C_GPS_Arduino_Library.h" and not the I2C library but it does not change the principle that much. The GPS module has its buffer, the Sparkfun library has its buffer and the I2C library has its buffer. That is, the GPS module buffer is cascaded into the Sparkfun library buffer via the I2C's buffer. Probably there are some edge cases in all that which are not properly handled.

17

Ah! That helps. I can get at the source code for the sparkfun library. If I were a betting man, which I’m not, I’d bet on the solution residing somewhere in the Sparkfun library.

If that’s so, where I’m at is a good place. I bought from Cool Components who are a respected reseller of Sparkfun hardware. They have escalated the problem with Sparkfun & I am awaiting their response. I am encouraged that Sparkfun will pay serious attention to this issue because Cool Components has more clout with them than me. I think the prospects are looking good and the buck is making its way slowly but surely to its rightful home … Sparkfun!

18

I have modified the test program to make it run with either the Sparkfun library or the Adafruit library. Complexity has been increased since the observed behaviour using the Sparkfun library is once data has become available a complete sentence can be captured, but using the Adafruit library, data may become unavailable in mid sentence.

A curiosity of the Adafruit library is that periodic "NULL" characters appear in the datastream. Since an NMEA sender never sends null characters, I have simply eliminated them from the messages.

At the top of the file is a #define statement, defining ADAFRUIT to select the Adafruit library, and commenting out the definition to select the Sparkfun library.

With this software my experience is about 25% messages fail when using the Sparkfun library. When I choose the Adafruit library, this drops to zero. I count that as a win.

// TWI GPS Dropout Test Program
// Written by Greg Walker 25-Aug-2023

#define ADAFRUIT
#ifdef ADAFRUIT
  #include <Adafruit_GPS.h>
#else
  #include "SparkFun_I2C_GPS_Arduino_Library.h"
#endif

const byte bMaxSentence = 85; // NMEA specifies 82 max
const long MONITOR_BAUDRATE = 2000000;

enum RUNOUT {BODY = 0, STAR, UPPER, LOWER, OVERRUN};

char NMEAsentence[bMaxSentence];
byte bNMEAindex;
RUNOUT eEndState;


#ifdef ADAFRUIT //Hook object to the library
  Adafruit_GPS myI2CGPS(&Wire);
#else
  I2CGPS myI2CGPS; 
#endif


unsigned long ulMessageCount;
unsigned long ulFailCount;

/////////////////////////////////////////////////////////////////////////////////////////////////////////////

void setup() 
{
  String strConfigString;

  ulMessageCount = NULL;
  ulFailCount = NULL;

  bNMEAindex = NULL;

  Serial.begin(MONITOR_BAUDRATE);
  // I2C_SPEED_STANDARD = 100000, I2C_SPEED_FAST = 400000
  #ifdef ADAFRUIT
    myI2CGPS.begin(0x10);
  #else
    if (!myI2CGPS.begin(Wire, I2C_SPEED_FAST)) 
    {
      Serial.println("Module failed to respond. Please check wiring.");
      for(;;);  // Hang forever
    }
  #endif

  Serial.println("GPS module found!");

  #ifdef ADAFRUIT
    myI2CGPS.sendCommand("PMTK220,100");
    delay(1000);
    myI2CGPS.sendCommand("PMTK314,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0");
  #else
    strConfigString = myI2CGPS.createMTKpacket(220, ",100"); // 100 for 10Hz, 1000 for 1Hz
    myI2CGPS.sendMTKpacket(strConfigString); // Set Update Rate
    delay(1000);
    strConfigString = myI2CGPS.createMTKpacket(314, ",0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0");
    myI2CGPS.sendMTKpacket(strConfigString); // Just enable $GNRMC messages
  #endif
  delay(1000);
}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////

void loop() 
{
  while(myI2CGPS.available())
  {
    if (bNMEAindex > NULL) // We're processing a message
    {
      if (CaptureSentence())
      {
        ReportSentence();
      }
    }
    else // We're searching for leading $
    {
      ScanForDollar();
    }
  }
}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////


bool CheckSentence(char OneLine[])
{
  bool bReply = false;
  byte bChecksumCalc = 0;
  size_t  nLength = strlen(OneLine);
  const char  HexList[] = "0123456789ABCDEF";

  if ((nLength >= 4) && (OneLine[0] == '$') && (OneLine[nLength - 3] == '*') && (nLength <= bMaxSentence))
  {
    byte bColumn;

    for(bColumn = 1; bColumn < (nLength - 3); bColumn++)
    {
      bChecksumCalc ^= OneLine[bColumn];
    }

    if((HexList[bChecksumCalc >> 4]) == (OneLine[nLength - 2]))
    {
      if ((HexList[bChecksumCalc & 0x0F]) == (OneLine[nLength - 1]))
      {
        bReply = true;
      }
    }
  }

  if(!bReply)
  {
    Serial.print("Error ");
    Serial.print(HexList[bChecksumCalc >> 4]);
    Serial.print(HexList[bChecksumCalc & 0x0F]);
    Serial.print(" ");
    Serial.println(nLength);
  }
 
  return bReply;
}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////

void ScanForDollar()
{
  if (myI2CGPS.read() == '$')
  {
    NMEAsentence[bNMEAindex++] = '$';
    eEndState = BODY;
  }
}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////

bool ReportSentence()
{
  bool bReply;
  double dfFailRatePercent;

  bReply = CheckSentence(NMEAsentence);
  if (!bReply) ulFailCount++;
  ulMessageCount++;

  dfFailRatePercent = ulFailCount * 100.0 / ulMessageCount;
  Serial.print(dfFailRatePercent);
  Serial.print(" ");
  Serial.print(ulMessageCount);
  Serial.print(" ");
  Serial.println(NMEAsentence);
}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////

bool CaptureSentence()
{
  bool bReply = false;
  
  if(bNMEAindex >= (bMaxSentence - 1)) eEndState = OVERRUN;
  
  switch (eEndState)
  {
    case BODY:
      NMEAsentence[bNMEAindex] = myI2CGPS.read();
      if(NMEAsentence[bNMEAindex] == NULL) break;
      //Serial.print(NMEAsentence[bNMEAindex]);
      if (NMEAsentence[bNMEAindex++] == '*')
      {
        eEndState = STAR;
      }
      break;

    case STAR:
      NMEAsentence[bNMEAindex++] = myI2CGPS.read();
      if(NMEAsentence[bNMEAindex - 1] == NULL)
      {
        bNMEAindex--;
        break;
      }
      //Serial.print(NMEAsentence[bNMEAindex - 1]);
      eEndState = UPPER;
      break;

    case UPPER:
      NMEAsentence[bNMEAindex++] = myI2CGPS.read();
      if(NMEAsentence[bNMEAindex - 1] == NULL)
      {
        bNMEAindex--;
        break;
      }
      //Serial.print(NMEAsentence[bNMEAindex - 1]);
      //Serial.print("    >");
      eEndState = LOWER;
      break;

    default: // LOWER or OVERRUN
      NMEAsentence[bNMEAindex++] = NULL;
      bNMEAindex = NULL;
      bReply = true;
  }

  return bReply;
}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////
19

The good news is the problem has been tracked down to an error in the Sparkfun GPS library. Using the Adafruit library is a fix. Checksum errors drop from about 25% to zero.

The bad news is that whilst the test progam compiles and runs without problem on a vanilla Arduino Nano (Atmega328P), this is a bit light on RAM for what I'm trying to do. Unfortunately, whilst my original test program (using Sparkfun library ) compiled fine with Nano and with Nano Every (Atmega4809), when I invoke the Adafruit library, it will compile and run for the basic Arduino Nano, it throws a wobbly when I switch personality to Arduino Nano Every. Here's the compiler diagnostic output.

In file included from c:\Users\User\OneDrive\GregData\Projects\Arduino\Data\Target Firmware\libraries\Adafruit_GPS_Library\src/Adafruit_GPS.h:66:0,
                 from c:\Users\User\OneDrive\GregData\Projects\Arduino\Data\Target Firmware\libraries\Adafruit_GPS_Library\src\Adafruit_GPS.cpp:31:
C:\Users\User\AppData\Local\Arduino15\packages\arduino\hardware\megaavr\1.8.8\libraries\Wire\src/Wire.h: In member function 'char Adafruit_GPS::read()':
C:\Users\User\AppData\Local\Arduino15\packages\arduino\hardware\megaavr\1.8.8\libraries\Wire\src/Wire.h:64:12: note: candidate 1: size_t TwoWire::requestFrom(int, int, int)
     size_t requestFrom(int, int, int);
            ^~~~~~~~~~~
C:\Users\User\AppData\Local\Arduino15\packages\arduino\hardware\megaavr\1.8.8\libraries\Wire\src/Wire.h:62:12: note: candidate 2: virtual size_t TwoWire::requestFrom(uint8_t, size_t, bool)
     size_t requestFrom(uint8_t, size_t, bool);
            ^~~~~~~~~~~
Sketch uses 16432 bytes (33%) of program storage space. Maximum is 49152 bytes.
Global variables use 3611 bytes (58%) of dynamic memory, leaving 2533 bytes for local variables. Maximum is 6144 bytes.

Has anyone had success with Adafruit GPS library to link Nano Every to a GPS using I2C/TWI connection? This uses the Wire library. I have happily used Arduino Nano Every with other TWI devices using the Wire library, and I have connected to the GPS with Sparkfun GPS library & Wire library.

Incidentally. At 3611 bytes of RAM used, this does seem a bit greedy. It's a "mere(!)" 1437 bytes for basic Arduino Nano.

20

I have just made an amazing discovery about the GPS - and I believe this will apply to the UBLOX receiver too, because that also uses the MT3333 engine.

I believe I had read that the receiver incorporates a buffer of 500 or 1000 bytes. I think I have read both figures but didn't capture references. To me, that was a prize worth having, particularly in the context of the puny 2k bytes of RAM in an Atmega 328. My conclusion is it's a lot better than that. Capturing 10 Hz fix data, I have been able to accumulate 6.5 seconds of historic data in the GPS and collect that reliably at my leisure. That's 4.6 k bytes of NMEA sentences stored in the GPS waiting for me to ask for it. I'm not even convinced I've reached the limit of its capability. Of course, the GPS might be saving that data in a more compressed form, requiring less storage, but that isn't the point. To me, that's a huge buffer store awaiting my convenience to be collected.

A further discovery is the Arduino can eat the data faster than the GPS can feed its output buffer. I'm finding that I can take about five sentences from the GPS before it tells me it has run out. I can then walk away from it for 50msec or so, and when I come back, I can get another five sentences - eating away at the backlog.

Of course, this is no use if I want to know where it is right now, but it's a tremendous asset if I just want to log the fix data. It gives me a huge margin to go away & play with other peripherals such as a GUI screen &/or an SD file store. I'm not saying I would ever expect to gather fix data that's six seconds old, but if I do ever get held up because I'm busy doing something else, I am comforted by the fact that the historic data isn't lost forever.