Reading Multiple Lines from SD Card

I am attempting to read values from my SD card using the adafruit SD breakout. The file contains a string of two words on the same line, but I would like to read multiple lines at once in the future. As of now, running the program prints a single spurious character and then ends. Any ideas? Thanks!

#include <SPI.h>
#include <SD.h>
Sd2Card card;
SdVolume volume;
SdFile root;

boolean go = true;
String userInput;
boolean pool;

void setup() {
  Serial.begin(9600);
  Serial.setTimeout(4500);
  pinMode(10, OUTPUT);
  digitalWrite(10, HIGH);
  Serial.println("Searching for SD Card...");
  if (!SD.begin(10)) {
    Serial.println("Cannot find SD card.");
  }
  while (!Serial) {                                                      // Open serial communications and wait for port to 
    ;
  }
    delay(100);
}

void loop() {
  if (go == true) {
    Serial.println("\nInitializing SD card...");
    if (!card.init(SPI_HALF_SPEED, 10)) {                                  //***10 used to be: chipSelect***
      Serial.println("initialization failed.");
      go = false;
      return;
    }
    else {
      Serial.println("Card Initialized");
    }

    Serial.println("File name (no extension):");
    userInput = Serial.readString();
    if (userInput == "") {
      Serial.println("Too slow, restart!");
      go = false;
      return;
    }
    userInput.trim();
    userInput.concat(".TXT");
    char fileName[userInput.length() + 1];
    userInput.toCharArray(fileName, userInput.length() + 1);
    File userFile = SD.open(fileName, FILE_WRITE);
    if (userFile) {
      userFile.close();
      SD.open(fileName, FILE_READ);
      if (!fileName) {
        Serial.println("Error opening file");
      }
      while (Serial.available()) {
        Serial.write(userFile.read());
      } 
      userFile.close();
      Serial.println("**End**");
      go = false;
      return;
    }
  }
}

There isn't really an easy way to do it, but how I have done it in the past (actually quite recently) is you need a counter to keep track of the lines and you need to know where the linefeed characters are.

In order to find the linefeed characters, you need to read the file character by character and compare the current character to '\n'. '\n' indicates you've reach the end of that line, so if you know where the line ends, you can figure out where the next one starts.

See reply #6.
http://forum.arduino.cc/index.php?topic=387994.msg2675259#msg2675259

I made that sketch to delete a line(s) from a file, but it can also be adapted to read a line(s) too.

Thanks for your reply, I'll be able to check it out at about this time next Monday if you're available.

Monday is not good for me, perhaps Tuesday.

Tuesday would be great.

Using the ShowFileContents method from your sketch, I was able to get the file to print. However, the program is now continuing to loop the contents of the file, even after userFile.available() returns 0. I feel as though it has something to do with being located within loop(), although I've set an additional while loop to control it that I have used in multiple other programs. This is the loop() method:

void loop() {
  while (go = true) {
    File userFile = SD.open("temp.txt", FILE_READ);
    Serial.println(userFile);
    if (userFile)
    {
      while (userFile.available()) {
        Serial.write(userFile.read());
      }
      Serial.println();
      userFile.close();
      Serial.println("done");
      go = false;
    }
  }
  ;
}

Any ideas as to what may be causing the looping? println tests show that it is never reaching the semicolon before the end. Thanks!

Nevermind, equals signs are hard. Thanks for the help!

:smiley:

Im working on making actual readLine functions. My end result is to add them to the SD library and make them core functions for everyone to use. Then again I could just make it another library and leave the SD library alone.

I'll come back when it is done and decided.

Ok, I have decided to make this its own extension library. However in the meantime, here is the sketch I have been working on.

The sketch is messy but the library won't be.

/*
 Demonstrate functions to copy, scan, read and delete lines from a file

 The circuit:
 * SD card attached to SPI bus as follows:
 ** MOSI - pin 11
 ** MISO - pin 12
 ** CLK - pin 13
 ** CS - pin 4

 created   March 29 2016
 by Andrew Mascolo

 */

#include <SPI.h>
#include <SD.h>

#define isSpace(x) (x == ' ')
#define isComma(x) (x == ',')
template<class T> inline Print &operator <<(Print &str, T arg)
{
  str.print(arg);
  return str;
}

struct LinesAndPositions
{
  int NumberOfLines; // number of lines in file
  int SOL[50]; // start of line in file
  int EOL[50]; // end of line in file
};

template<typename T, size_t N>
char* ReadSelectedLinesFromFile(T (&buf)[N], char * filename, char * StrOfLines)
{
  char tempBuf[N];
 
  Serial << "Reading multiple lines, please wait";
  for (unsigned short i = 0, j = strlen(StrOfLines), index = 0; i <= j; i++)
  {
    char C = (*StrOfLines++);
    if (isComma(C) || (i == j))
    {
      ReadLineFromFile(tempBuf, filename, index);
      strncat(buf, tempBuf, N);
      index = 0;
    }
    else if (isSpace(C)) continue;
    else
      index = (index * 10) + C - '0';
    if ((i % 2) == 0)
      Serial << ".";
  }
  
  Serial.println();
  return buf;
}

void setup()
{
  // Open serial communications and wait for port to open:
  Serial.begin(115200);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  Serial.print("Initializing SD card...");
  // On the Ethernet Shield, CS is pin 4. It's set as an output by default.
  // Note that even if it's not used as the CS pin, the hardware SS pin
  // (10 on most Arduino boards, 53 on the Mega) must be left as an output
  // or the SD library functions will not work.
  pinMode(53, OUTPUT);

  if (!SD.begin(53)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  // Uncomment this if you already have a copy of COPY.txt
  CopyFiles("test.txt", "COPY.txt");

  ShowFileContents("test.txt");

  // Uncomment this if you don't already have a copy of COPY.txt
  //CopyFiles("COPY.txt", "test.txt");
  //ShowFileContents("COPY.txt");

  LinesAndPositions X = FindLinesAndPositions("test.txt");
  Serial << "\nNumber of lines from test.txt:" << X.NumberOfLines << "\n";
  Serial << "--------------------------------\n\n";
  
  char buf[50] = {NULL};
  //ReadLineFromFile(buf, "test.txt", 1) << "\n";
  //ReadMultipleLinesFromFile(buf, "test.txt", 1, 2);
  ReadSelectedLinesFromFile(buf, "test.txt", "1,3,4");
  Serial << "From passed in buffer:\n" << buf << "\n";
  
  Serial << "--------------------------------\n";
  
  //DeleteLineFromFile("test.txt", 2); // delete line 2 from file
  //DeleteMultipleLinesFromFile("test.txt", 2, 3); delete lines 2 through 3 from file
  DeleteSelectedLinesFromFile("test.txt", "1, 3, 4"); // delete individual lines 1,3,4 from file. **Takes a few seconds**
  Serial.println("Done.");

  ShowFileContents("test.txt");
  X = FindLinesAndPositions("test.txt");
  Serial << "\nNumber of lines from test.txt:" << X.NumberOfLines << "\n";

}

void loop()
{
  // nothing happens after setup
}

struct LinesAndPositions FindLinesAndPositions(char* filename)
{
  File myFile;
  LinesAndPositions LNP;

  myFile = SD.open(filename);
  if (myFile)
  {
    LNP.NumberOfLines = 0;
    LNP.SOL[0] = 0; //the very first start-of-line index is always zero
    
    unsigned int Pos;
    while (myFile.available())
    {
      Pos = myFile.position(); // get the position. Put before .read(), otherwise it will not start at 0 like it should.
      if (myFile.read() == '\n') // read until the newline character has been found
      {
        LNP.EOL[LNP.NumberOfLines] = Pos; // record the location of where it is in the file
        LNP.NumberOfLines++; // update the number of lines found
        LNP.SOL[LNP.NumberOfLines] = Pos + 1; // the start-of-line is always 1 character more than the end-of-line location
      }
    }
    LNP.EOL[LNP.NumberOfLines] = Pos + 1; // record the last locations
    LNP.NumberOfLines += 1; // record the last line

    myFile.close();
  }

  return LNP;
}

void CopyFiles(char* ToFile, char* FromFile)
{
  File myFileOrig;
  File myFileTemp;

  if (SD.exists(ToFile))
    SD.remove(ToFile);

  //Serial << "\nCopying Files. From:" << FromFile << " -> To:" << ToFile << "\n";

  myFileTemp = SD.open(ToFile, FILE_WRITE);
  myFileOrig = SD.open(FromFile);
  if (myFileOrig)
  {
    while (myFileOrig.available())
    {
      myFileTemp.write( myFileOrig.read() ); // make a complete copy of the original file
    }
    myFileOrig.close();
    myFileTemp.close();
    //Serial.println("done.");
  }
}

void ShowFileContents(char* filename)
{
  Serial.println(filename);
  File myFile = SD.open(filename);
  if (myFile)
  {
    while (myFile.available()) {
      Serial.write(myFile.read());
    }
    Serial.println();
    myFile.close();
  }
}

void DeleteLineFromFile(char * filename, int Line)
{
  DeleteMultipleLinesFromFile(filename, Line, Line);
}

void DeleteSelectedLinesFromFile(char * filename, char * StrOfLines)
{
  byte offset = 0;
  Serial << "Deleting multiple lines, please wait";
  for (unsigned short i = 0, j = strlen(StrOfLines), index = 0; i <= j; i++)
  {
    char C = (*StrOfLines++);
    if (isComma(C) || (i == j))
    {
      DeleteLineFromFile(filename, index - offset);
      offset++;
      index = 0;
    }
    else if (isSpace(C)) continue;
    else
      index = (index * 10) + C - '0';
    if ((i % 2) == 0)
      Serial << ".";
  }
  Serial.println();
}

void DeleteMultipleLinesFromFile(char * filename, int SLine, int ELine)
{
  File myFileOrig;
  File myFileTemp;

  // If by some chance it exists, remove the temp file from the card
  if (SD.exists("tempFile.txt"))
    SD.remove("tempFile.txt");

  // Get the start and end of line positions from said file
  LinesAndPositions FileLines = FindLinesAndPositions(filename);

  if ((SLine > FileLines.NumberOfLines) || (ELine > FileLines.NumberOfLines))
    return;

  myFileTemp = SD.open("tempFile.txt", FILE_WRITE);
  myFileOrig = SD.open(filename);

  if (myFileOrig)
  {
    unsigned int position = 0;
    char C;
    
    while (myFileOrig.available())
    {
      position = myFileOrig.position();
      C = myFileOrig.read();

      // Copy the file but exclude the entered lines by user
      if ((position < FileLines.SOL[SLine - 1]) || (position > FileLines.EOL[ELine - 1]))
        myFileTemp.write(C);
    }
    myFileOrig.close();
    myFileTemp.close();
  }

  //copy the contents of tempFile back to the original file
  CopyFiles(filename, "tempFile.txt");

  // Remove the tempfile from the SD card
  if (SD.exists("tempFile.txt"))
    SD.remove("tempFile.txt");
}

char* ReadLineFromFile(char* Buffer, char* filename, const byte Line)
{
  return ReadMultipleLinesFromFile(Buffer, filename, Line, Line);
}

char* ReadMultipleLinesFromFile(char* Buffer, char* filename, byte SLine, byte ELine)
{
  SLine -= 1;
  ELine -= 1;
  
  File myFile = SD.open(filename);
  if (myFile)
  {
    LinesAndPositions FileLines = FindLinesAndPositions(filename);
    
    // go to beginning of line
    myFile.seek(FileLines.SOL[SLine]);
   
    while (myFile.available())
    {
      unsigned int Pos = myFile.position();
      if (Pos <= (FileLines.EOL[ELine]))
      {
        Buffer[Pos - FileLines.SOL[SLine]] = myFile.read();
        Buffer[Pos - FileLines.SOL[SLine] + 1] = '\0';
      }
      else
        break;
    }
    
    myFile.close();
    return Buffer;
  }
  return 0;
}

TEST.TXT (24 Bytes)

COPY.TXT (63 Bytes)

OK and here is the library. I will put it on my github account after I am sure it is the best I can get it.

Please keep in mind it is memory heavy, I will work on its memory consumption and hopefully get it down to a reasonable amount.

Let me know if you encounter any issues.

Apparently I uploaded the library with the wrong example sketch. Fixed

SD_Extension.zip (4.39 KB)

HazardsMind:
OK and here is the library. I will put it on my github account after I am sure it is the best I can get it.

Please keep in mind it is memory heavy, I will work on its memory consumption and hopefully get it down to a reasonable amount.

Let me know if you encounter any issues.

Apparently I uploaded the library with the wrong example sketch. Fixed

HarzardsMind i got a error message

invalid operands of types 'char*' and 'const char [2]' to binary 'operator<<'
when i use one of your example

Ex.
SDE.ReadLineFromFile(buf, "five.txt", 1) << "\n";

can i use variable to change the value of the position

for example like this

i = 1;

SDE.ReadLineFromFile(buf, "five.txt", i) << "\n";

when i did this i got problem