TinyGPS encode doesn't work at the same time as SD write

here's my project: read nmea data from Serial3, then write it to a log on an sd card. simple, easy. i got this up and running, i open the log file, write bytes until i see a line feed, break the loop, and write the buffered data to the file. the trouble comes when i wanted to add a real time display, so i started using TinyGPS. ajust before i'd add data to the file buffer i'd encode the byte, and the if it returns true i'd call a function to update the lcd. now, the encode function ALWAYS returns false. but, if i comment out the write function, it works as expected. it doesn't matter if i run the encode function from a parent function, or if i run the write function from a parent function. i've attached simplified code to demonstrate the bug.

#include <SD.h>
#include <TinyGPS.h>

TinyGPS gps;
File logFile;

void setup() {
  Serial.begin(9600);
  Serial3.begin(9600);
  pinMode(53,OUTPUT);
  if(!SD.begin(4)) while(1);
}

void loop() {
  int line = 1;
  logFile = SD.open("logfile.log",FILE_WRITE);
  while(1) {
    if(Serial3.available()) {
      char v = Serial3.read();
      if(gps.encode(v)) Serial.println("New data");
      logFile.write(v);
      if(v == 10) break;
    }
  }
  logFile.close();
}

The write() call is after the decode() call so I can't imagine how that would affect it. Are you absolutely positive you have described the symptoms correctly? If so, the only thing I can think of is that you're suffering some sort of memory corruption. Have you checked the free RAM?

The gps.stats() method might give you a clue as to why the decode() is failing.

If so, the only thing I can think of is that you're suffering some sort of memory corruption. Have you checked the free RAM?

The fact that OP is using Serial3 would tend to rule out, in this simple code, running out of memory as a cause of the difficulties.

PaulS:

If so, the only thing I can think of is that you're suffering some sort of memory corruption. Have you checked the free RAM?

The fact that OP is using Serial3 would tend to rule out, in this simple code, running out of memory as a cause of the difficulties.

On the basis he must be using a Mega?

On the basis he must be using a Mega?

It's the only one with more than two serial ports, so, yes, that was my assumption.

PeterH:
Are you absolutely positive you have described the symptoms correctly?

the encode function works when i comment out the write function, and it doesn't when i don't. i'll look at ram usage, is there some other testing i should try?

On the basis he must be using a Mega?

i am indeed using a mega 2560

mckennie:
is there some other testing i should try?

Can you use Serial.println() to log the sequence of characters passed to gps.encode(), and see whether the sequence changes when the logFile.write() statement is included?

It's only a stab in the dark, but I suppose that if the call to logFile.write() took a significant time then the gps input stream might overflow its buffer. One possible solution in that case would be to buffer the received line in a char array until the terminating character was recognised, and then log the whole line in one go.

PeterH:
Can you use Serial.println() to log the sequence of characters passed to gps.encode(), and see whether the sequence changes when the logFile.write() statement is included?

why Serial.println()? wouldn't Serial.print() give an easier to read output?

It's only a stab in the dark, but I suppose that if the call to logFile.write() took a significant time then the gps input stream might overflow its buffer. One possible solution in that case would be to buffer the received line in a char array until the terminating character was recognised, and then log the whole line in one go.

i don't think that's it, but when i get around to testing i'll time my loops with and without it. when i first wrote the thing i wanted to buffer the line in an array but i was afraid of over running the array, as i don't know the maximum possible length of an nmea sentance.

when i first wrote the thing i wanted to buffer the line in an array but i was afraid of over running the array, as i don't know the maximum possible length of an nmea sentance.

Guess, if you can't find it on-line. Defend yourself from buffer overruns by throwing away anything too long. Try 100 to start with. Alternatively, write a sketch that echoes the NMEA data to serial and observe until you're convinced you've seen all the sentences your device uses.

and observe until you're convinced you've seen all the sentences your device uses.

And then add a margin of safety AND buffer overflow checking.

wildbill:
Defend yourself from buffer overruns by throwing away anything too long

that never even crossed my mind, always glad to be learning new stuff :smiley:

wildbill:
Alternatively, write a sketch that echoes the NMEA data to serial and observe until you're convinced you've seen all the sentences your device uses.

hadn't thought of this either, i'll give it a go.

sorry i can't run any tests, my arduino is tied up in a project right now. i should be able to run all of them tomorrow evening.

I'm having the same problem when using SD and TinyGPS my Arduino Mega 1280 always crash after some catch data from gps.
my code:

gps.f_get_position(&flat, &flon, &age);  
gps.f_altitude();
gps.f_speed_kmph();
...

//Convert float to string and save in var
SD_lat
SD_log
SD_ALT
...
//concatenate variables
GPS_dataString = SD_lat + ";"+ SD_log + ";" +SD_ALT + ... ;

//Save in SD  
  File dataFile = SD.open("BOATLOG.csv", FILE_WRITE);
    if (dataFile)
    {
  
      dataFile.print(GPS_dataString);//escreve no cartao pulando uma linha
      dataFile.print(";");
      dataFile.print(TEMP_dataString);  
      dataFile.println();
      dataFile.close();
    }
    else
    {
        Serial.println("\nCouldn't open the log file!");
    }
GPS_dataString = SD_lat + ";"+ SD_log + ";" +SD_ALT + ... ;

Is GPS_dataString a resource leeking/memory corrupting String?

      dataFile.print(SD_lat);
      dataFile.print(";");
      dataFile.print(SD_log);
      dataFile.print(";");
      dataFile.print(SD_ALT);
      ...

Does the same thing with no dynamic memory allocation/free/heap corruption.

PeterH:
Can you use Serial.println() to log the sequence of characters passed to gps.encode(), and see whether the sequence changes when the logFile.write() statement is included?

everything comes though fine

It's only a stab in the dark, but I suppose that if the call to logFile.write() took a significant time then the gps input stream might overflow its buffer. One possible solution in that case would be to buffer the received line in a char array until the terminating character was recognised, and then log the whole line in one go.

i switched to the buffering method, and the problem is still there. the print function always takes less than 15 milliseconds

If you have checked and double checked that the sequence in identical in the normal and failing cases (including carriage return, newline or whatever other characters the GPS receiver may be sending you) then this indicates that the problem is not caused by dropped characters or corruption on the input stream so there's no point looking for buffering, timing or overflow type errors. The problem must be occurring within the encode() function. From the symptoms you describe, I suppose that the SD.write call might be affecting the internal state of the gps object, causing the subsequent parsing to fail.

Have you tried calling gps.stats() to see whether it gives you any clues about the nature of the errors?

To simplify the test code even further, can you hard-code a sample NMEA string in the sketch (so excluding the GPS unit itself and all the Serial3 reading from the problem) and see whether you can reproduce the problem, then confirm that commenting out the SD write makes the identical encode succeed?

PeterH:
From the symptoms you describe, I suppose that the SD.write call might be affecting the internal state of the gps object, causing the subsequent parsing to fail.

the parsing isn't subsequent, it comes before. (in the context of a single iteration)

Have you tried calling gps.stats() to see whether it gives you any clues about the nature of the errors?

i'll try this now

To simplify the test code even further, can you hard-code a sample NMEA string in the sketch (so excluding the GPS unit itself and all the Serial3 reading from the problem) and see whether you can reproduce the problem, then confirm that commenting out the SD write makes the identical encode succeed?

and this after that.

mckennie:
the parsing isn't subsequent, it comes before. (in the context of a single iteration)

Obviously, the call to sd.write() isn't going to retrospectively affect the results of parsing that has already been done. But it could affect the result of subsequent parsing i.e. parsing of subsequent characters.

PeterH:
Obviously, the call to sd.write() isn't going to retrospectively affect the results of parsing that has already been done. But it could affect the result of subsequent parsing i.e. parsing of subsequent characters.

mckennie:
in the context of a single iteration

I think it is time for you to make a change to the TinyGPS library. In the encode() method, Serial.print() the stored sentence, and any intermediate values that it produces, to see WHY it thinks the sentence is never valid.

Is the sentence getting corrupted? Is some other value getting overwritten? You simply are not gathering enough information yet.