Make sure you've got the latest version of the Library, v4.2.4. You can do this in the Library Manager.
how to check the "track1" check if it is FALSE or TRUE and "free" the "memory / flash" before checking "track2 and track3"?
You can't free FLASH memory. It is part of the uploaded program. You can't free the program space used by a function, either.
The only "extra" RAM bytes are the startPt and endPt copies of each segment in a track. They are local variables, so they are "freed" before that function returns.
do you think that maybe a delay would do you think that maybe a delay would solve it to "clear" the memory/flash?
No.
delay does not magically fix things. Never use delay.
I previously needed approximately 900 coordinates, but with your code this got dramatically smaller.
Just to confirm, you are using my latest examples that show tracks with endpoints of straight segments, NOT all the intermediate points along a straight segment.
As the D1 Mini PRO has more Flash (16Megabytes), it would be possible to have for example 50 thousand coordinates
50,000 endpoints would use about 400,000 bytes. There is plenty of room in 16MB.
However, there are limits in C++ about how large a single array can be. The limits are different for different processors. For 8-bit AVR processors (ATmega328 in an UNO/Nano, or ATmega2560), the limit is 32768 bytes (4096 endpoints per track). I do not know the limit for ESP8266 MCUs.
Is there a limit to the number of coordinates?
You can fit 2 million locations in 16MB. But...
I should also mention SPIFFS (SPI FLASH File System) in the ESP8266 family. You can use the FLASH memory like a file system. In Your case, I would recommend binary files with one track per file. These files would not have the same limitation as C++ arrays.
You would "read" them with the SPIFFS library, NOT with pgm_read_xxx calls.
I tried to put approximately 15 thousand coordinates in total (separated in the 5 tracks of 3 in 3 thousand) and gave this error:
I would suggest trying smaller array sizes to see if you are running into some limit.
Posting your code might be a good idea. :-/
{-25.9850000,-49.6651200},
These are supposed to be the integer forms:
{-259850000,-496651200},
I don't think that would cause a problem, but they will get truncated to { -25, -49 }. Not very accurate.
Hmm... The ESP8266 supports the double floating-point type, which is 8 bytes. So a pair would use 16 bytes, and 3000 pairs would use 48000 bytes. That might be over a limit. But I don't see how this could be the problem. The use of a double literal for a int32_t initializer shouldn't cause a problem.
Try limiting each track to 2000 endpoints to make sure it's not an array size problem. Try a few arrays with 2040, 2047 and 2048 endpoints to see if it crosses the magic 32768 bytes.
Also, the pgm_read_ptr was not implemented correctly on the ESP8266. Make sure you're up-to-date.
When I use for example 1300 coordinates on each track (5 tracks in total - 6500 coordinates) it works perfectly. But when that number goes from 7 thousand he presents that error
Have you tried the integer forms?
When I use for example 1300 coordinates on each track (5 tracks in total - 6500 coordinates) it works perfectly. But when that number goes from 7 thousand he presents that error
1300 coordinates in each track = 10400 bytes per track
times 5 = 50400 bytes
1400 coordinates in each track = 11200 bytes per track
times 5 = 56000 bytes
There's nothing magic about those numbers, but there is an issue on 8-bit AVRs with memory addresses bigger than 64k. Perhaps the program size plus the flash size is crossing the 65535 magic number boundary. I don't know what the issues are on an ESP8266 with 16-bit (0..65535, aka "near") vs 32-bit addresses (aka "far"). Perhaps there's a different compiler option that builds with "far" as the default addressing. Maybe it does this already?
Looking back at the error message, I see this:
Soft WDT reset
Have you enabled the WatchDog Timer? Is it enabled by default?
If testing a long track takes a long time, you may not reset the WDT quickly enough.
Sorry, I just don't know what kinds of things can go wrong on the ESP family of MCUs.
There is an ESP exception decoder here. It might be able to tell you where the exception happened (if it's not just a WDT reset).
If it's just a WDT reset, it can happen at any part of the sketch.
Image embedded for our convenience:
Technique described here.
And now you can see why we want you to paste the text into your post, in a code block. -_- Just select the text in the window and copy it into your post. The image doesn't have all of the stack contents, just back to the Print::write address. This could be misleading, though... That could be an uninitialized variable inside the distanceFromptToLine routine. It is probably from a previous print call.
After downloading the image and opening it in a separate window, it looks like it was in the sin function when "something" happened.
Because the sin function simply calculates some values from the argument, I suspect that there is some kind of issue with the stack memory. There's no way for a math function to cause an exception like this. The stack does not look very deep, so I doubt that it is a stack overflow.
This trace is different from your previous post, so I assume that it crashes in different places every time.
Sorry, that's all I can tell from the trace. I would research what kinds of issues you might have with lots of FLASH data on the ESP8266, or switch to using SPIFFS.
Could it be some incompatibility of the NeoGPS library with the "D1 Mini PRO 16MB - ESP8266" card I'm using?
Since you only have a problem when there is a lot of FLASH data, it is probably not a NeoGPS problem.
I have also seen problems when users try to put some of the NeoGPS configuration files in the sketch directory. NeoGPS programs must be configured by H files in the Libraries/NeoGPS/src directory only. Make sure there are no extra files in your sketch directory.
how I could do to have each track loaded into a ".txt" file
The basic differences from using FLASH are
* Initialize SPIFFS in setup
* Every time you receive a new GPS fix with a valid location, iterate through the SPIFFS directory to get each filename.
* For each filename, open it, read each line of characters to get a point, test that point, then close the file.
* For comparison, each line of characters from the file must be converted (parsed) into a Location_t.
Once you can load a Location_t from the file, all the other code stays the same.
Maybe something like this:
#include <SPIFFS.h> ???
#include <SoftwareSerial.h>
static const int RXPin = 4, TXPin = 3;
static const uint32_t GPSBaud = 9600;
#include <NMEAGPS.h>
SoftwareSerial gpsPort(RXPin, TXPin);
const int LED = 15;
NMEAGPS gps; // Parse
gps_fix fix; // GPS information structure
int fixCount = 0; // Number of fixes received (also a seconds count)
const float MARGEKM = 0.111; // km = 111m
bool onTrack = false;
// Forward declaration of the point-to-line distance functions
float distanceFromPtToLine
( const NeoGPS::Location_t & pt,
FileType & file,
float threshold )
float distanceFromPtToSegment
( const NeoGPS::Location_t & pt,
const NeoGPS::Location_t & startPt,
const NeoGPS::Location_t & endPt );
void setup()
{
Serial.begin(9600);
gpsPort.begin(9600);
pinMode(LED, OUTPUT );
digitalWrite(LED, LOW);
--> Initialize SPIFFS
} // setup
void loop()
{
// Process GPS characters
if (gps.available( gpsPort )) {
// Once per second a new fix structure is ready.
fix = gps.read();
fixCount++;
// Check the track distance once every 5 seconds
if (fixCount >= 5) {
fixCount = 0; // reset counter
Serial.println( F("Acquired Data") ); // F macro saves RAM
Serial.println( F("-------------") );
// If the GPS knows the position, compare it to the track points
bool onTrack = false;
if (fix.valid.location) {
Serial.print ( fix.latitude(), 6 );
Serial.print ( ',' );
Serial.println( fix.longitude(), 6 );
// for (size_t i=0; i < TRACK_COUNT; i++) {
for (each filename in root directory) {
FileType track = next file;
float distance = distanceFromPtToLine( fix.location, track, MARGEKM );
Serial.print( F("Track") );
Serial.print( i+1 );
Serial.print( F(" distance = ") );
Serial.println( distance, 6 );
if (distance < MARGEKM) {
onTrack = true;
break;
}
}
} else {
// No location available (yet).
Serial.println( '?' );
}
updateTrackLED( onTrack );
}
}
} // loop
// ASCII C string to point
// Parse one location from a NUL-terminated character array
//
NeoGPS::Location_t atoPt( char *cstring )
{
NeoGPS::Location_t RAMloc;
char *field = strtok( cstring, ", " );
RAMloc.lat( atol( field ) ); // Expects the integer form
field = strtok( nullptr, ", " );
RAMloc.lon( atol( field ) );
return RAMloc;
} // atoPt
float distanceFromPtToLine
( const NeoGPS::Location_t & pt,
FileType & file,
float threshold )
{
file.open( READONLY );
// Read lines of ASCII characters (a C string) from the file.
// Convert each cstring to a point for comparison.
char cstring[20];
size_t numRead = file.fgets( cstring, sizeof(cstring) );
if (numRead <= 0)
return NAN; // empty file
NeoGPS::Location_t startPt( atoPt( cstring ) );
float minDistance = pt.DistanceKm( startPt );
float distance = minDistance;
size_t i = 1;
while (minDistance > threshold) {
// Read another line of characters from the file
numRead = file.fgets( cstring, sizeof(cstring) );
if (numRead <= 0)
break; // no more
NeoGPS::Location_t endPt( atoPt( cstring ) );
distance = distanceFromPtToSegment( pt, startPt, endPt );
// Keep the closest approach
if (minDistance > distance) {
minDistance = distance;
}
i++;
if (i >= n)
break; // no more segments
// For next iteration, use the end point as the new start point
startPt = endPt;
}
file.close();
return minDistance;
} // distanceFromPtToLine
float distanceFromPtToSegment
( const NeoGPS::Location_t & pt,
const NeoGPS::Location_t & startPt,
const NeoGPS::Location_t & endPt )
{
float distance;
int32_t dLat = endPt.lat() - startPt.lat();
int32_t dLon = endPt.lon() - startPt.lon();
if ((dLat == 0) && (dLon == 0)) {
// Line segment is degenerate (a single point).
// Just calculate a point-to-point distance.
distance = pt.DistanceKm( startPt );
} else {
// Calculate projection of vector pt-start onto pt-end
float norm2 = (float)dLat * (float)dLat + (float)dLon * (float)dLon;
float ptToStartLat = pt.lat() - startPt.lat();
float ptToStartLon = pt.lon() - startPt.lon();
float t = (ptToStartLat * dLat) + (ptToStartLon * dLon);
// t/norm2 is a 0.0 to 1.0 fraction along the segment at
// which pt is "closest" to the segment. The vector from
// pt to this point is perpendicular to the segment.
// Therefore, the distance from pt to the segment is
// simply the distance from pt to this closest approach.
if (t < 0.0) {
// Closest approach is somewhere before start point,
// just use the distance to the start point.
distance = pt.DistanceKm( startPt );
} else if (t < norm2) {
// Closest approach is between the start and end point.
t /= norm2; // between 0.0 and 1.0
NeoGPS::Location_t closest
(
startPt.lat() + (int32_t)(t * dLat),
startPt.lon() + (int32_t)(t * dLon)
);
distance = pt.DistanceKm( closest );
} else { // t >= norm2
// Closest approach is somewhere after end point,
// just use the distance to the end point.
distance = pt.DistanceKm( endPt );
}
}
return distance;
} // distanceFromPtToSegment
void updateTrackLED( bool onOff )
{
if (onOff) {
Serial.println("True");
digitalWrite(LED, HIGH);
} else {
Serial.println("False");
digitalWrite(LED, LOW);
}
} // updateTrackLED
Cheers,
/dev
Line 73 looks weird. A programming editor can help you find mismatched { }. Notepad++ is very popular.
Ok, here's some more untested code for you to try:
#include <FS.h>
#include <SoftwareSerial.h>
static const int RXPin = 4, TXPin = 3;
static const uint32_t GPSBaud = 9600;
#include <NMEAGPS.h>
SoftwareSerial gpsPort(RXPin, TXPin);
const int LED = 15;
NMEAGPS gps; // Parse
gps_fix fix; // GPS information structure
int fixCount = 0; // Number of fixes received (also a seconds count)
const float MARGEKM = 0.111; // km = 111m
bool onTrack = false;
// Forward declaration of the point-to-line distance functions
float distanceFromPtToLine
( const NeoGPS::Location_t & pt,
File & file,
float threshold );
float distanceFromPtToSegment
( const NeoGPS::Location_t & pt,
const NeoGPS::Location_t & startPt,
const NeoGPS::Location_t & endPt );
void setup()
{
Serial.begin(9600);
gpsPort.begin(9600);
pinMode(LED, OUTPUT );
digitalWrite(LED, LOW);
// Initialize SPIFFS
SPIFFS.begin();
} // setup
void loop()
{
// Process GPS characters
if (gps.available( gpsPort )) {
// Once per second a new fix structure is ready.
fix = gps.read();
fixCount++;
// Check the track distance once every 5 seconds
if (fixCount >= 5) {
fixCount = 0; // reset counter
Serial.println( F("Acquired Data") ); // F macro saves RAM
Serial.println( F("-------------") );
// If the GPS knows the position, compare it to the track points
bool onTrack = false;
if (fix.valid.location) {
Serial.print ( fix.latitude(), 6 );
Serial.print ( ',' );
Serial.println( fix.longitude(), 6 );
//}
// for (size_t i=0; i < TRACK_COUNT; i++) {
//for (each filename in root directory) {
Dir dir = SPIFFS.openDir("/");
while (dir.next()) {
Serial.print(dir.name());
File track = dir.openFile("r");
Serial.println(track.size());
float distance = distanceFromPtToLine( fix.location, track, MARGEKM );
track.close();
Serial.print( F("Track") );
//Serial.print( i+1 );
Serial.print( F(" distance = ") );
Serial.println( distance, 6 );
if (distance < MARGEKM) {
onTrack = true;
break;
}
}
} else {
// No location available (yet).
Serial.println( '?' );
}
updateTrackLED( onTrack );
}
}
}// loop
// ASCII C string to point
// Parse one location from a NUL-terminated character array
//
NeoGPS::Location_t atoPt( char *cstring )
{
NeoGPS::Location_t RAMloc;
char *field = strtok( cstring, ", " );
RAMloc.lat( atol( field ) ); // Expects the integer form
field = strtok( nullptr, ", " );
RAMloc.lon( atol( field ) );
return RAMloc;
} // atoPt
int16_t readLine( File & file, char* str, int16_t num, char* delim = nullptr )
{
char ch;
int16_t n = 0;
while (file.available() && ((n + 1) < num)) {
ch = read();
if (ch == 26)
break;
str[n++] = ch;
if (!delim) {
// delete CR
if (ch == '\r') {
continue;
}
if (ch == '\n') {
break;
}
} else {
if (strchr(delim, ch)) {
break;
}
}
}
str[n] = '\0';
return n;
} // readLine
float distanceFromPtToLine
( const NeoGPS::Location_t & pt,
File & file,
float threshold )
{
// Read lines of ASCII characters (a C string) from the file.
// Convert each cstring to a point for comparison.
char cstring[32];
size_t numRead = readLine( file, cstring, sizeof(cstring) );
if (numRead <= 0)
return NAN; // empty file
NeoGPS::Location_t startPt( atoPt( cstring ) );
float minDistance = pt.DistanceKm( startPt );
float distance = minDistance;
size_t i = 1;
while (minDistance > threshold) {
// Read another line of characters from the file
numRead = readLine( file, cstring, sizeof(cstring) );
if (numRead <= 0)
break; // no more
NeoGPS::Location_t endPt( atoPt( cstring ) );
distance = distanceFromPtToSegment( pt, startPt, endPt );
// Keep the closest approach
if (minDistance > distance) {
minDistance = distance;
}
i++;
if (i >= n)
break; // no more segments
// For next iteration, use the end point as the new start point
startPt = endPt;
}
return minDistance;
} // distanceFromPtToLine
float distanceFromPtToSegment
( const NeoGPS::Location_t & pt,
const NeoGPS::Location_t & startPt,
const NeoGPS::Location_t & endPt )
{
float distance;
int32_t dLat = endPt.lat() - startPt.lat();
int32_t dLon = endPt.lon() - startPt.lon();
if ((dLat == 0) && (dLon == 0)) {
// Line segment is degenerate (a single point).
// Just calculate a point-to-point distance.
distance = pt.DistanceKm( startPt );
} else {
// Calculate projection of vector pt-start onto pt-end
float norm2 = (float)dLat * (float)dLat + (float)dLon * (float)dLon;
float ptToStartLat = pt.lat() - startPt.lat();
float ptToStartLon = pt.lon() - startPt.lon();
float t = (ptToStartLat * dLat) + (ptToStartLon * dLon);
// t/norm2 is a 0.0 to 1.0 fraction along the segment at
// which pt is "closest" to the segment. The vector from
// pt to this point is perpendicular to the segment.
// Therefore, the distance from pt to the segment is
// simply the distance from pt to this closest approach.
if (t < 0.0) {
// Closest approach is somewhere before start point,
// just use the distance to the start point.
distance = pt.DistanceKm( startPt );
} else if (t < norm2) {
// Closest approach is between the start and end point.
t /= norm2; // between 0.0 and 1.0
NeoGPS::Location_t closest
(
startPt.lat() + (int32_t)(t * dLat),
startPt.lon() + (int32_t)(t * dLon)
);
distance = pt.DistanceKm( closest );
} else { // t >= norm2
// Closest approach is somewhere after end point,
// just use the distance to the end point.
distance = pt.DistanceKm( endPt );
}
}
return distance;
} // distanceFromPtToSegment
void updateTrackLED( bool onOff )
{
if (onOff) {
Serial.println("True");
digitalWrite(LED, HIGH);
} else {
Serial.println("False");
digitalWrite(LED, LOW);
}
} // updateTrackLED
The major changes:
-
To display the name of each file, use
dir.name(a char *), notdir.fileName(a dreadedString). -
If you open each track file before
distanceFromPtToLine, you should close it after the function, not inside the function. -
I added a
readLinefunction for reading one text line from a file. It is derived from the SdFat fgets function.
Good luck.
Could you tell me what could be wrong?
Well, the only thing I can see is in atoPt:
RAMloc.lat( atol( field ) ); // Expects the integer form
If your text file contains floating -point numbers like this:
-19.8828400,-42.1371000
-19.8831500,-42.1373500
-19.8835200,-42.1375600
... you must parse them with atof:
RAMloc.lat( atof( field ) ); // Expects the floating-point form
Do the same thing for RAMloc.lon
Maybe you should write a test program that does not use GPS. It should print all the locations it loads from the SPIFFs. Print each filename, print each line you read, then print each location that gets parsed from that line.
This will tell you if the problem is in reading the directory, reading a line or parsing a line. The rest of the sketch (distance functions) has been tested several times.
Here's the test program I was suggesting. It uses parts of your complete program to show what it reads from SPIFFS.
#include <FS.h>
#include <Location.h>
// forward declarations for utility functions
NeoGPS::Location_t atoPt( char *cstring );
size_t readLine( File & file, char* str, size_t num, const char * delim = nullptr );
void setup()
{
Serial.begin(9600);
SPIFFS.begin();
Dir dir = SPIFFS.openDir("/");
while (dir.next()) {
File file = dir.openFile("r");
// Print each file name and size
Serial.print( file.name() );
Serial.print( ' ' );
Serial.println( file.size() );
while (file.available()) {
char cstring[32];
size_t numRead = readLine( file, cstring, sizeof(cstring) );
// Print each line
Serial.println( cstring );
if (numRead <= 0) {
Serial.println( "------------" );
break;
}
NeoGPS::Location_t startPt( atoPt( cstring ) );
// Print the values that were parsed from that line
Serial.print( " --> " );
Serial.print( startPt.lat() );
Serial.print( ',' );
Serial.println( startPt.lon() );
}
file.close();
}
}
void loop() {}
// ASCII C string to point
// Parse one location from a NUL-terminated character array
//
NeoGPS::Location_t atoPt( char *cstring )
{
NeoGPS::Location_t RAMloc;
char *field = strtok( cstring, ", " );
RAMloc.lat( atol( field ) ); // Expects the integer form
field = strtok( nullptr, ", " );
RAMloc.lon( atol( field ) );
return RAMloc;
} // atoPt
size_t readLine( File & file, char* str, size_t num, const char * delim )
{
char ch;
size_t n = 0;
while (file.available() && ((n + 1) < num)) {
ch = file.read();
if (ch == 26)
break;
str[n++] = ch;
if (!delim) {
// delete CR
if (ch == '\r') {
continue;
}
if (ch == '\n') {
break;
}
} else {
if (strchr(delim, ch)) {
break;
}
}
}
str[n] = '\0';
return n;
} // readLine
If this shows only 2 lines per file, there could be a problem with (1) your txt files, (2) the readLine function, or (3) the FS library.
do you believe that it is also working normally because you have normally accessed the files in SIPFFS?
Yes, the file functions and the file contents are correct.
There must be a problem in the distance functions. You should print the distance it calculates for each point:
distance = distanceFromPtToSegment( pt, startPt, endPt );
Serial.print( i );
Serial.print( " distance = " );
Serial.println( distance, 6 );
Make sure the distance values make sense for the track. Try adding your location to the file to make sure the distance is small.
If the calculated distances are off, there is a problem in distanceFromPtToSegment. I'd have to double-check the math.
The distance calculation looks pretty close. With these two locations:
-14.852242, -40.835194
-14.8464600, -40.8354500
... this site calculates 0.6435.
But it looks like it is comparing only one location from track1.
It should only stop comparing locations if the distance is less than the threshold:
while (minDistance > threshold) {
The threshold is 0.111, so why would 0.560 stop the comparison? What is breaking out of the loop?
Hint: There are 2 bad lines in your sketch. They were also in the original "untested code" I gave you for the SPIFFS version (back in reply 71).
I commented the "break" and it worked perfectly!
You should comment the "if", too:
// if (i >= numRead)
// break; // no more segments
Otherwise, the "if" will apply to the next statement after the comment break.
It will act like you wrote this:
if (i >= numRead)
startPt = endPt;
That is not what you want.