Reading only text fields from CSV file

Hi, I’m having a problem reading text fields from a CSV file and can’t find the solution, hoping someone can help.

I first started with the readCSV example from the sdFat32 library. In the example they have 3 field types for 4 columns, text, float and long being read. When I use the example it works fine.

When I try and read only 3 text fields I run into problem. In order for it to work correctly I need to have the 3rd field followed by a comma even though nothing is coming next. I can easily add that so it works correctly but the thing I don’t like is if I open that csv file in excel and save it, excel will remove those commas as they are pointless in it’s view. Once those are removed I can’t get the code to read it properly. Here’s some code explaining the situation.

This chuck of code works great as long as the 3rd column has a comma following it. If it doesn’t it’ll read the first row correctly however in the 3rd column of first row it’ll put the 2nd rows 1st column, because there is no comma it doesn’t know to stop. Once that happens an error comes and it won’t continue.

  if (myFile) 
  {
    float data;  // data
    char col1[60];  // temp variable to pass incoming data
    char c1;  // space for commas.
            
    ifstream sdin(grDay); // open input file 
    if (!sdin.is_open()) {Serial.println("can't open file");} // can't open file
  
    // read file
    while (1) {       
      sdin.get(col1, sizeof(col1), ','); // Get 1st text field.
      sdin >> c1;
      sendCol1 = col1;

      sdin.get(col1, sizeof(col1), ','); // Get 2nd text field.
      sdin >> c1;
      sendCol2 = col1;

      sdin.get(col1, sizeof(col1), ','); // Get 3rd text field.
      sdin >> c1;   // This line is why that unneeded comma is required but I can't get anything else to work. :(                           
      sendCol3 = col1;
      
      if (sdin.fail()) {break;} // if format fail   
//    sdin >> c1 >> data; // Get data from file   
      sdin.skipWhite(); // Skip blank space
      if (sdin.fail()) {break;} // end of file
    }
    // Error in an input line
    if (!sdin.eof()) {Serial.println("can't read file");}    
    myFile.close();
  }

Now in the code above, 8 lines up, you’ll see one line commented out, if I add a 4th column to the csv with a float or int type it’ll work as expected without having a comma following the 4th column.

So I’m baffled why it doesn’t do this is the last column is a text field. Any help would be appreciated.

No experience with the specific library or methods.

sdin >> c1;   // This line is why that unneeded comma is required but I can't get anything else to work. :(

What have you tried? Have you tried to leave that line out?

Your example does not seem to be based on ReadCsv but on ReadCsvStream; modyfying line 48 of that example to only read 2 columns should probably do the trick

  while (1) {
    // Get text field.
    sdin.get(sendCol1, sizeof(text), ',');

    // Assume EOF if fail.
    if (sdin.fail()) {
      break;
    }
    // Get commas and numbers.
    sdin >> c1 >> sendCol2 >> c2 >> sendCol3;

    ...
    ...

Thanks for the reply, it could be the readCsvStream, I don’t have that example so maybe mine is in fact that, I haven’t changed much, just tried all combos and my first is the only way I can make it work but it’s not right, been trying every combo possible but it’s being really stubborn.

The code you posted was what I tried first but doesn’t work and I’ve done almost ever combo I can. I’ve tried without the GET and just used that line you highlighted with all 3 fields, doing that it’ll get most of the data and finish but some characters are missing. In my original code I also tried without that line you mentioned but no go, I also tried with just sdin; which uploads but no go. Doesn’t make sense why it doesn’t know the line ends with a text field.

Here’s an example of what I get in the serial monitor with your suggestion. The lines should be Line 1,test,test2 as you can see in the write function. Notice the first comma is missing and it grabs row 2 but drops the “L”. Whatever I try I get different combinations like below.

Line 1test,test2ine
2test,test2ine
3test,test2ine
error: bad input
/*
 *  This example reads a simple CSV, comma-separated values, file.
 *  Each line of the file has a label and three values, a long and two floats.
 */
#include <SPI.h>
#include <SdFat.h>

// SD chip select pin
const uint8_t chipSelect = 53;

// file system object
SdFat sd;

// create Serial stream
ArduinoOutStream cout(Serial);

char fileName[] = "testfile20.csv";
//------------------------------------------------------------------------------
// store error strings in flash to save RAM
#define error(s) sd.errorHalt(F(s))
//------------------------------------------------------------------------------
// read and print CSV test file
void readFile() {
  long lg;
  float f1, f2;
  char text[10];
  char text2[10];
  char text3[10];
  char c1, c2, c3;  // space for commas.

  // open input file
  ifstream sdin(fileName);

  // check for open error
  if (!sdin.is_open()) {
    error("open");
  }

  // read until input fails
  while (1) {
    // Get text field.
    sdin.get(text, sizeof(text), ',');

    // Assume EOF if fail.
    if (sdin.fail()) {
      break;
    }

    // Get commas and numbers.
    sdin >> c1 >> text2 >> c2 >> text3;

    // Skip CR/LF.
    sdin.skipWhite();

    if (sdin.fail()) {
      error("bad input");
    }

    // error in line if not commas
  //  if (c1 != ',' || c2 != ',') {
 //     error("comma");
 //   }

    // print in six character wide columns
    cout << text << text2 << text3 << endl;
  }
  // Error in an input line if file is not at EOF.
  if (!sdin.eof()) {
    error("readFile");
  }
}
//------------------------------------------------------------------------------
// write test file
void writeFile() {

  // create or open and truncate output file
  ofstream sdout(fileName);

  // write file from string stored in flash
  sdout << F(
          "Line 1,test,test2\n"
          "Line 2,test,test2\n"
          "Line 3,test,test2\n"
          "Line 4,test,test2\n") << flush;

  // check for any errors
  if (!sdout) {
    error("writeFile");
  }

  sdout.close();
}
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(115200);

  // initialize the SD card at SPI_HALF_SPEED to avoid bus errors with
  // breadboards.  use SPI_FULL_SPEED for better performance
  if (!sd.begin(chipSelect, SPI_HALF_SPEED)) {
    sd.initErrorHalt();
  }

  // create test file
  writeFile();

  cout << endl;

  // read and print test
  readFile();

  cout << "\nDone!" << endl;
}
void loop() {}

Have a look at the parse example in Serial Input Basics

...R

Your filename is too long. It should be at most 8 characters (excluding the extension). Read up on 8.3 filename

Robin2:
Have a look at the parse example in Serial Input Basics

…R

Thanks, I went through it but not able to see a solution.

arduino_new:
Your filename is too long. It should be at most 8 characters (excluding the extension). Read up on 8.3 filename

Unfortunately that’s not the problem, I have no issues reading long file/folder names. I did just change it just in case but still same results. I can read files no problem just can’t see the end/start of a CSV row.