How to read or select a certain rows of data in csv file?

Hi, everyone.

I have searched and look at the previous post about read/write data in the csv file. But none that I've searched talk about selecting or read certain rows of data. In my coding, I used a comma to make the time and data separated.

Can anyone suggest something for me? A link maybe? So I can search more.
I have the data perfectly write to the csv file, I can read all the data in it, but I can't call a certain data that I need.

void dDisplay()
{
  // open the file:
  File dataFile = SD.open("datalog.csv", FILE_WRITE);
  Serial.print(hour());
  printDigits(minute());
  printDigits(second());
  Serial.print(' ');
  Serial.print(day());
  Serial.print(' ');
  Serial.print(month());
  Serial.print(' ');
  Serial.print(year());
  Serial.print(' ');
  Serial.print(A0);
  Serial.println();
  // if the file is available, write to it:
  if (dataFile) {
    dataFile.print(hour());
    dataFile.print('/');
    dataFile.print(minute());
    dataFile.print('/');
    dataFile.print(second());
    dataFile.print(' ');
    dataFile.print(",");
    dataFile.print(day());
    dataFile.print(':');
    dataFile.print(month());
    dataFile.print(':');
    dataFile.print(year());
    dataFile.print(' ');
    dataFile.print(",");
    dataFile.print(A0);
    dataFile.close();
  }
  else {
    // if the file didn't open, print an error:
    Serial.println("error opening datalog.csv");
  }  
}

Please anyone.
Thanks.

SyukriY:
Hi, everyone.

I have searched and look at the previous post about read/write data in the csv file. But none that I've searched talk about selecting or read certain rows of data. In my coding, I used a comma to make the time and data separated.

Can anyone suggest something for me? A link maybe? So I can search more.
I have the data perfectly write to the csv file, I can read all the data in it, but I can't call a certain data that I need.

You must understand your data file is a simple "text" file. If you need to search for specific entries in a random sequence, then read the data into an array, then you can access any entry by index. With "0" being the first entry.

Paul

There are two basic ways to lay data down in a file: 1) sequential, 2) random access. It sounds like you have written the data as a sequential file. Searching will likely be confined to reading the file byte-by-byte until you find what you need. Random access usually requires that each piece of data occupies the same number of bytes, called a record. If your file has 100 records, it looks like a line of 100 bricks laid end-to-end. If each brick is, say, 7 bytes long and you want to "read" the 33rd brick, you can pick up the read head and skip over 32 bricks (e.g., 224 bytes) and drop it down to read the 33rd brick. Sequential files are "dense", in that the data are packed tightly against each other so there's no wasted space, but random access files can be search quickly if the record position has some inherent meaning. A little Googling should give you enough to decide and implement the code.

Adding to @econjack’s reply

csv files also contain records :wink: Each record ends (semi officially) with (see https://www.ietf.org/rfc/rfc4180.txt). So you need to read byte-by-byte and count the combinations to get to the correct record.

If you use a fixed-width format instead of comma separated, all records have the same length and you can easily access a record by seeking N*M (N is the record number, M is the size of the record).

By the way, your datafile might not contain what you expect; usually A0 is the pin number and your data file will contain the pin number and not the value that is read from that pin.

PS
In a csv file, a field can contain the separator character (in your case the comma as well as line endings ); it does not seem to be the case in your situation but if you ever need to read a csv file generated by a spreadsheet program you may encounter this.

Thank you all for your reply.

If I understand correctly on what @econjack mention, data as a sequential file cannot randomly pick the data because the arrangement is close together. But, data as a random file can be freely accessed.
Therefore, I'll choose to write the data in random data file.

According to @Paul_KD7HB an array should do the trick to select a particular data that we need.
Is that, how to write data in random data file?

Please correct me if I'm wrong.

Here I give something that I've just write.

void setup(void)
{
  Serial.begin(9600);
  while (!Serial);
  setSyncProvider(RTC.get);   // the function to get the time from the RTC
  if (timeStatus() != timeSet)
    Serial.println("Unable to sync with the RTC");
  else
    Serial.println("RTC has set the system time");

  Serial.print("Initializing SD card...");
  if (!SD.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");
  //Write Log File Header
  File dataFile = SD.open("datalog.csv", FILE_WRITE);
  if (dataFile)
  {
    dataFile.println(", , ,"); //Just a leading blank line, incase there was previous data
    String header = "ID, Date, Time, AnalogReading";
    dataFile.println(header);
    dataFile.close();
    Serial.println(header);
  }
  else
  {
    Serial.println("Couldn't open datalog file");
  }
}

void dDisplay()
{
  Serial.print(hour());
  printDigits(minute());
  printDigits(second());
  Serial.print(' ');
  Serial.print(day());
  Serial.print(' ');
  Serial.print(month());
  Serial.print(' ');
  Serial.print(year());
  Serial.println();

 String dataString = String(id) + ", " + String(day(), month(), year()) + ", " + String(hour(), minute(), second()) + ", " + String(A0);
  // open the file:
  File dataFile = SD.open("datalog.csv", FILE_WRITE);
  // if the file is available, write to it:
  if (dataFile) {
    dataFile.println(dataString);
    dataFile.close();
  }
  else {
    // if the file didn't open, print an error:
    Serial.println("error opening datalog.csv");
  }  
}

The code is fine for me. Because the output is expected. But what command should I
use to read a certain data.

and @sterretje thank you for the link.
I’ve searched about and thus, I came out to used comma. :slight_smile:

If you have any more link that I can study about selecting data or about csv file, please let me know.

Thank you. :wink:

is carriage return (’\r’) and linefeed (’\n’).

A Serial.println() will take care of that while writing.

If you go the fixed width route, you can not directly print a number because it will require leading spaces or leading zeroes to get a fixed width.

Sorry, what do you mean by fixed width route?

SyukriY:
Sorry, what do you mean by fixed width route?

What econjack called random access. But anyway, your reply #4 indicates that you want to stick with comma separated values without having a fixed width.

SyukriY:
The code is fine for me. Because the output is expected. But what command should I
use to read a certain data.

Just use dataFile.read() to read a single character at a time. For the first record, you can simply read up to and store the characters in an array and next process it. If you want to get to the second record, read up to the first and throw it away and next read up to the second and store it in an array and next process it; and so on.