Need help logging my TF-Luna LiDAR sensor and Neo 6M GPS module to SD-Card using Arduino

I am currently trying to log both my LiDAR sensor data and my GPS data to an SD card. The code compiles fine, but when I upload it, the labels will get written into the SD card file but the data from both the LiDAR sensor and GPS will not show up in the file. I am using an Arduino Mega 2560, but I also have an Arduino UNO that I can quickly swap to if needed. This is my code:

#include <TinyGPS++.h>
#include <SoftwareSerial.h>
#include <SPI.h>
#include <SD.h>

TinyGPSPlus gps;

File myFile;

static const int RXPin = 4, TXPin = 3; //connect RX from GPS to pin 4 and TX from GPS to pin 5
static const uint32_t GPSBaud = 9600;
const int chipSelect = 53;
String Labels = "Latitude, Longitude, Speed(mph), Altitude(cm),";
int dist; //actual distance measurements of LIDAR
int check; //save check value
int i;
int uart[9]; //save data measured by LIDAR
const int HEADER = 0x59;

SoftwareSerial sL(6,7); //define software serial port name for LiDAR 
SoftwareSerial ss(RXPin, TXPin); //define software serial port name for GPS


void setup() {
  Serial.begin(9600);
  pinMode(chipSelect, OUTPUT);

  Serial.println("Initializating SD card...");
  if (!SD.begin()) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization successful.");

  //open the file, note that only one file can open at a time!
  myFile = SD.open("Data.csv", FILE_WRITE);

  //if the file opened okay write to it:
  if (myFile) {
    Serial.println("Writing labels to Data.csv...");
    myFile.println(Labels);
    myFile.close();
    Serial.println("done writing labels to Data.csv");
  }
  else {
    Serial.println("failed to write labels to Data.csv");
  }
  ss.begin(GPSBaud);
  Serial.begin(9600);
  sL.begin(115200);
}

void loop() {
  delay(1000);

  //Getting LiDAR data
  if (sL.available()) { //check if serial port has data input
    if (sL.read() == HEADER) { //assess data package frame header 0x59
      uart[0] = HEADER;
      if (sL.read() == HEADER) { //assess data package frame header 0x59
        uart[1] = HEADER;
        for (i = 2; i < 9; i++) { //save data in array
          uart[i] = ssL.read();
        }
        check = uart[0] + uart[1] + uart[2] + uart[3] + uart[4] + uart[5] + uart[6] + uart[7];
        if (uart[8] == (check & 0xff)) { //verify the recieved data as per protocol
          dist = uart[2] + uart[3] * 256; //calculate distance value
          Serial.print("Altitude(cm) = ");
          Serial.println(dist); //print distance from LiDAR to serial monitor
        }
      }
    }
  }
   
  //Getting GPS data
  while (ss.available() > 0) {
    gps.encode(ss.read());
    if (gps.location.isUpdated()) {
      Serial.print("Latitude = "); 
      Serial.println(gps.location.lat(), 6);
      Serial.print("Longitude = "); 
      Serial.println(gps.location.lng(), 6);
      Serial.print("Speed(mph) = ");
      Serial.println(gps.speed.mph(), 2);
    }
  }
  
  //Write GPS and LiDAR data to SD Card
  myFile = SD.open("Data.csv", FILE_WRITE);
  if (myFile) {
    myFile.print(gps.location.lat(), 6);
    myFile.print(",");
    myFile.print(gps.location.lng(), 6);
    myFile.print(",");
    myFile.print(gps.speed.mph(), 2);
    myFile.print(",");
    myFile.println(dist);
  }
   else {
      Serial.println("Failed to open Data.csv to write LiDAR and GPS data...");
   }
      myFile.close(); 
}

Please check the first topic telling how to get the best from this forum. That tells how to post code using code tags. Now it looks like text and is a lot less readable.
Try and add just a hard coded constant to the label when writing to the SD card.

Have You verified that the LIDAR as well as GPS are giving correct values? The simple example codes for LIDAR resp. GPS are well worth running to verify that as a beginning.

What shows up in the file after the "label"? What is "the label"? Filename?

Sorry, I fixed it so now my code is posted using code tags.

So I wrote two different programs before this one. The first one is logging the LiDAR sensor data to my SD card and it works flawlessly. The second one is logging the GPS data to my SD card and it also works flawlessly. This code I posted is the combination of the two codes, but for some reason, none of the LiDAR data nor the GPS data is getting logged to the SD card.

The file name that shows up in my SD card is Data.csv which is exactly what I want. The only thing though that is written in that file is the labels which I have identified in my code as:
String Labels = "Latitude, Longitude, Speed(mph), Altitude(cm),";

The SD card initialization seems to be fine. Basically what I concluded is that the void setup() part of my code runs fine but when it gets to the void loop(), nothing happens.

Okey. Logging LIDAR alone works as well as logging GPS alone works. Strange. Hmm. Do You use any delimiter, data separator, between the LIDAR and GPS data? Fixed format for Lat, Long, Speed and Altitude would of course work. Just You need to read the file the proper way.
One possible error could be when reading the registration.
Time is 04:30 Am and time, since long, goto bed.....

Not knowing what a delimiter and a data separator are... I do not believe I am using them. If a possible error can be when reading the registration how can I fix that if it is fixable?

I have been on this project for a week now and after looking over the code multiple times I cannot find what the issue is.

If You use a fix format like float, float, int, int You need to make sure that when reading the file the reading code sticks to that.
Delimiters are special characters separating the parameters from each other.¨
As an example, creating files for Excel, each data can be separated by a comma, a TAB character.
Any help?
Can You read and display the record length of the SD file? Do You close the file properly after writing?

Yes, it seems I can read and display values in the file in the SD card and I use myFile.close(); whenever I finish writing to the SD card.

I am using a 16gb Micro-SD so there should be enough space for the data to be written to the file.

I use commas to separate the labels and the values to a different cell in excel.

That looks all wright to me. I don't get it. You tell "but the data from both the LiDAR sensor and GPS will not show up in the file."

We are close to, or beyond, the boarder of my comfort zone now.
High time to sleep now...., for me.

Yeah, it is almost like the void loop() part of the code does not get executed and it just writes the labels in the file.

And no worries goodnight :slight_smile:

Use serial monitor and Serial.print to show what You send to the SD!
Nighty, nighty....

Your using two software serial interfaces on a Mega, thats not going to end well .....................

The Mega has 3 hardware serial ports, use them instead.

When you wrote a seperate test program for the Lidar, on the Mega presumably, did you use the same pins for the software serial for the Lidar ?

I suspect not ....................

To prove that something is being done and what that something is, use some serial.prints.

void loop() {

  delay(1000);
Serial.println( "delay(1000);" );
  //Getting LiDAR data
  if (sL.available()) { //check if serial port has data input
Serial.println( "if (sL.available())" );
    if (sL.read() == HEADER) { //assess data package frame header 0x59
Serial.println( "if (sL.read() == HEADER)" );
      uart[0] = HEADER;
      if (sL.read() == HEADER) { //assess data package frame header 0x59
Serial.println( "if (sL.read() == HEADER)" );
        uart[1] = HEADER;
        for (i = 2; i < 9; i++) { //save data in array
          uart[i] = ssL.read();
        }
        check = uart[0] + uart[1] + uart[2] + uart[3] + uart[4] + uart[5] + uart[6] + uart[7];
        if (uart[8] == (check & 0xff)) { //verify the recieved data as per protocol
Serial.println( "if (uart[8] == (check & 0xff))" );
          dist = uart[2] + uart[3] * 256; //calculate distance value
          Serial.print("Altitude(cm) = ");
          Serial.println(dist); //print distance from LiDAR to serial monitor
        }
      }
    }
  }
}

You can do the rest. After you've serial printed up the rest of the code. Post the changed code. Let us know which lines do not print? Oh, the lines that do print, remove the serial prints, you know that the code is getting to that point .

So for the test programs for LiDAR and the GPS logging to the SD card I actually programmed them for the Arduino UNO and they work fine with connections to the same ports. Now I changed my code to work with my Arduino UNO and it seems that the distance from the LiDAR is being logged finally, but the GPS module still is not getting logged... here is my new code using the UNO:

#include <TinyGPS++.h>
#include <SoftwareSerial.h>
#include <SPI.h>
#include <SD.h>

TinyGPSPlus gps;

File myFile;

static const int RXPin = 7, TXPin = 6; //connect RX from GPS to pin 6 and TX from GPS to pin 7
static const uint32_t GPSBaud = 9600;
const int chipSelect = 10;
String Labels = "Latitude, Longitude, Speed(mph), Altitude(cm),";
int dist; //actual distance measurements of LIDAR
int check; //save check value
int i;
int uart[9]; //save data measured by LIDAR
const int HEADER = 0x59;

SoftwareSerial Serial1(2,3); //define software serial port name for LiDAR as Serial1 and define pin2 as RX and pin3 as TX
SoftwareSerial ss(RXPin, TXPin); //define software serial port name for GPS


void setup() {
  Serial.begin(9600);
  pinMode(chipSelect, OUTPUT);

  Serial.println("Initializating SD card...");
  if (!SD.begin()) {
    Serial.println("initialization failed!");
    return;
  }
  else {
    Serial.println("initialization successful.");
  }

  //open the file, note that only one file can open at a time!
  myFile = SD.open("Data.csv", FILE_WRITE);

  //if the file opened okay write to it:
  if (myFile) {
    Serial.println("Writing labels to Data.csv...");
    myFile.println(Labels);
    myFile.close();
    Serial.println("done writing labels to Data.csv");
  }
  else {
    Serial.println("failed to write labels to Data.csv");
  }
  ss.begin(GPSBaud);
  Serial1.begin(115200);
}

void loop() {
  //delay(1000);

  //Getting LiDAR data
  if (Serial1.available()) { //check if serial port has data input
    if (Serial1.read() == HEADER) { //assess data package frame header 0x59
      uart[0] = HEADER;
      if (Serial1.read() == HEADER) { //assess data package frame header 0x59
        uart[1] = HEADER;
        for (i = 2; i < 9; i++) { //save data in array
          uart[i] = Serial1.read();
        }
        check = uart[0] + uart[1] + uart[2] + uart[3] + uart[4] + uart[5] + uart[6] + uart[7];
        if (uart[8] == (check & 0xff)) { //verify the recieved data as per protocol
          dist = uart[2] + uart[3] * 256; //calculate distance value
          Serial.print("Altitude(cm) = ");
          Serial.println(dist); //print distance from LiDAR to serial monitor
        }
      }
    }
  }
   
  //Getting GPS data
  while (ss.available() > 0) {
    gps.encode(ss.read());
    if (gps.location.isUpdated()) {
      Serial.print("Latitude = "); 
      Serial.println(gps.location.lat(), 6);
      Serial.print("Longitude = "); 
      Serial.println(gps.location.lng(), 6);
      Serial.print("Speed(mph) = ");
      Serial.println(gps.speed.mph(), 2);
    }
  }
  
  //Write GPS and LiDAR data to SD Card
  myFile = SD.open("Data.csv", FILE_WRITE);
  if (myFile) {
    myFile.print(gps.location.lat(), 6);
    myFile.print(",");
    myFile.print(gps.location.lng(), 6);
    myFile.print(",");
    myFile.print(gps.speed.mph(), 2);
    myFile.print(",");
    myFile.println(dist);
  }
   else {
      Serial.println("Failed to open Data.csv to write LiDAR and GPS data...");
   }
      myFile.close(); 
}

As I mentioned in my reply to srnet, I changed my code to work with my UNO instead of the Mega and it seems to be successfully logging the LiDAR data now, but the GPS data is still not getting log not sure why now. Here is my code:

#include <TinyGPS++.h>
#include <SoftwareSerial.h>
#include <SPI.h>
#include <SD.h>

TinyGPSPlus gps;

File myFile;

static const int RXPin = 7, TXPin = 6; //connect RX from GPS to pin 6 and TX from GPS to pin 7
static const uint32_t GPSBaud = 9600;
const int chipSelect = 10;
String Labels = "Latitude, Longitude, Speed(mph), Altitude(cm),";
int dist; //actual distance measurements of LIDAR
int check; //save check value
int i;
int uart[9]; //save data measured by LIDAR
const int HEADER = 0x59;

SoftwareSerial Serial1(2,3); //define software serial port name for LiDAR as Serial1 and define pin2 as RX and pin3 as TX
SoftwareSerial ss(RXPin, TXPin); //define software serial port name for GPS


void setup() {
  Serial.begin(9600);
  pinMode(chipSelect, OUTPUT);

  Serial.println("Initializating SD card...");
  if (!SD.begin()) {
    Serial.println("initialization failed!");
    return;
  }
  else {
    Serial.println("initialization successful.");
  }

  //open the file, note that only one file can open at a time!
  myFile = SD.open("Data.csv", FILE_WRITE);

  //if the file opened okay write to it:
  if (myFile) {
    Serial.println("Writing labels to Data.csv...");
    myFile.println(Labels);
    myFile.close();
    Serial.println("done writing labels to Data.csv");
  }
  else {
    Serial.println("failed to write labels to Data.csv");
  }
  ss.begin(GPSBaud);
  Serial1.begin(115200);
}

void loop() {
  //delay(1000);

  //Getting LiDAR data
  if (Serial1.available()) { //check if serial port has data input
    if (Serial1.read() == HEADER) { //assess data package frame header 0x59
      uart[0] = HEADER;
      if (Serial1.read() == HEADER) { //assess data package frame header 0x59
        uart[1] = HEADER;
        for (i = 2; i < 9; i++) { //save data in array
          uart[i] = Serial1.read();
        }
        check = uart[0] + uart[1] + uart[2] + uart[3] + uart[4] + uart[5] + uart[6] + uart[7];
        if (uart[8] == (check & 0xff)) { //verify the recieved data as per protocol
          dist = uart[2] + uart[3] * 256; //calculate distance value
          Serial.print("Altitude(cm) = ");
          Serial.println(dist); //print distance from LiDAR to serial monitor
        }
      }
    }
  }
   
  //Getting GPS data
  while (ss.available() > 0) {
    gps.encode(ss.read());
    if (gps.location.isUpdated()) {
      Serial.print("Latitude = "); 
      Serial.println(gps.location.lat(), 6);
      Serial.print("Longitude = "); 
      Serial.println(gps.location.lng(), 6);
      Serial.print("Speed(mph) = ");
      Serial.println(gps.speed.mph(), 2);
    }
  }
  
  //Write GPS and LiDAR data to SD Card
  myFile = SD.open("Data.csv", FILE_WRITE);
  if (myFile) {
    myFile.print(gps.location.lat(), 6);
    myFile.print(",");
    myFile.print(gps.location.lng(), 6);
    myFile.print(",");
    myFile.print(gps.speed.mph(), 2);
    myFile.print(",");
    myFile.println(dist);
  }
   else {
      Serial.println("Failed to open Data.csv to write LiDAR and GPS data...");
   }
      myFile.close(); 
}

Your now using two software serial interfaces on a UNO, thats not going to end well either .....................

You can use two software serial interfaces on a UNO but only one can be listening at a time, see the reference;

https://www.arduino.cc/en/Reference/softwareSerial

I see. What if I end the two serials after using them? For example, I put Serial1.end() after I get LiDAR data and put ss.end() after I get GPS data

Try it and see ?

So I tried it and it still shows the same thing. I would like to add that the LiDAR sensor data does not always get logged it randomly logs one piece of data once in a while so not sure why the lag. Here is my code:

#include <TinyGPS++.h>
#include <SoftwareSerial.h>
#include <SPI.h>
#include <SD.h>

TinyGPSPlus gps;

File myFile;

static const int RXPin = 7, TXPin = 6; //connect RX from GPS to pin 6 and TX from GPS to pin 7
static const uint32_t GPSBaud = 9600;
const int chipSelect = 10;
String Labels = "Latitude, Longitude, Speed(mph), Altitude(cm),";
int dist; //actual distance measurements of LIDAR
int check; //save check value
int i;
int uart[9]; //save data measured by LIDAR
const int HEADER = 0x59;

SoftwareSerial Serial1(2,3); //define software serial port name for LiDAR as Serial1 and define pin2 as RX and pin3 as TX
SoftwareSerial ss(RXPin, TXPin); //define software serial port name for GPS


void setup() {
  Serial.begin(9600);
  pinMode(chipSelect, OUTPUT);

  Serial.println("Initializating SD card...");
  if (!SD.begin()) {
    Serial.println("initialization failed!");
    return;
  }
  else {
    Serial.println("initialization successful.");
  }

  //open the file, note that only one file can open at a time!
  myFile = SD.open("Data.csv", FILE_WRITE);

  //if the file opened okay write to it:
  if (myFile) {
    Serial.println("Writing labels to Data.csv...");
    myFile.println(Labels);
    myFile.close();
    Serial.println("done writing labels to Data.csv");
  }
  else {
    Serial.println("failed to write labels to Data.csv");
  }
  Serial.end();
}

void loop() {
  delay(1000);

  //Getting LiDAR data
  Serial.begin(9600);
  Serial1.begin(115200);
  if (Serial1.available()) { //check if serial port has data input
    if (Serial1.read() == HEADER) { //assess data package frame header 0x59
      uart[0] = HEADER;
      if (Serial1.read() == HEADER) { //assess data package frame header 0x59
        uart[1] = HEADER;
        for (i = 2; i < 9; i++) { //save data in array
          uart[i] = Serial1.read();
        }
        check = uart[0] + uart[1] + uart[2] + uart[3] + uart[4] + uart[5] + uart[6] + uart[7];
        if (uart[8] == (check & 0xff)) { //verify the recieved data as per protocol
          dist = uart[2] + uart[3] * 256; //calculate distance value
          Serial.print("Altitude(cm) = ");
          Serial.println(dist); //print distance from LiDAR to serial monitor
        }
      }
    }
  }
  Serial1.end();
  Serial.end();
   
  //Getting GPS data
  ss.begin(GPSBaud);
  Serial.begin(9600);
  while (ss.available() > 0) {
    gps.encode(ss.read());
    if (gps.location.isUpdated()) {
      Serial.print("Latitude = "); 
      Serial.println(gps.location.lat(), 6);
      Serial.print("Longitude = "); 
      Serial.println(gps.location.lng(), 6);
      Serial.print("Speed(mph) = ");
      Serial.println(gps.speed.mph(), 2);
    }
  }
  ss.end();
  //Write GPS and LiDAR data to SD Card
  myFile = SD.open("Data.csv", FILE_WRITE);
  if (myFile) {
    myFile.print(gps.location.lat(), 6);
    myFile.print(",");
    myFile.print(gps.location.lng(), 6);
    myFile.print(",");
    myFile.print(gps.speed.mph(), 2);
    myFile.print(",");
    myFile.println(dist);
  }
   else {
      Serial.println("Failed to open Data.csv to write LiDAR and GPS data...");
  }
  myFile.close();
  Serial.end(); 
}

I noticed though when I compile my code it says "Low memory available, stability problems may occur." Could this be the issue why the GPS data is not getting logged?

Can you post the output from the Serial Monitor when the program runs?

Do these statements get executed?

          Serial.print("Altitude(cm) = ");
          Serial.println(dist); //print distance from LiDAR to serial monitor

Also - when you say the data does not get written to the file, do you at least get the field delimiters (the "," characters that seperate the fields) written?

Couple of other points:

  • 115200 is pretty fast for Software Serial - it is unreliable at higher speeds.
  • the way you are reading the serial data is not reliable.
 if (sL.available()) { //check if serial port has data input

.available() returns a number of bytes... you should be checking for > 0.

Also, you can't assume the just because this returns a value > 0, that all the data has arrived in the serial buffer. There are some good posts on here about how to read serial data correctly.

  • Your code seems to be looking for 2 header bytes of 0x59 from the GPS - is this intended?
  • I would suggest putting some more Serial.print statements in your code so you can confirm where it is going wrong.