Using Arduino SD, does the pointer remain in place after file.close()?

I created a text file (string.txt) with the following format:

string1\n
\n
string2\n
\n
...

I also wrote a function which returns the i'th string in the text file (it does so by counting the endlines). Here is the stub:

char* getString(char file[], int i);

The problem is that I can't make consecutive, multiple calls to getString() without the file.open() returning "false". For instance:

getString("string.txt", 1);  // This is ok.

but

getString("string.txt", 1); 
getString("string.txt", 2); //This causes open() to return false.

I worked under the assumption that after closing a file, reopenning the same file causes the byte pointer (from which read() pulls a byte) to point to the very beginning (and not point at the last thing that it pointed to before closing.

Attached my code in case you wanted to try it out.

/*
 * Note to Self: 
 *   1. The formal parameter of an arduino function should be a char[] if you wish to pass a string in "" (quotes form)
        into the function.
     2. strcpy must be passed a "" (empty string in quotes) to "clear" the string; Entering anything into  
           strcpy(quote, "<breaks program>"); 
        will break the program. 
     3. char* quote = (char*)malloc(sizeof(char)*800); 
        Mallocing is hard coded by longest quote we've in .txt file (we found upper bound). 
        We need a method to find the size in bytes of the quote. Also, it seems that we can't physically 
        appropriate more than ~800+ bytes for characters. The program will store the string until it can't,
        crashes, and then reboots. This limits how big of a quote we can have!
     4. Remember to free() after malloc()!
 */ 

#include <string.h>
#include <SPI.h>
#include <SD.h>

#define DEBUG 0

char *getQuote(char txtFile[], int quoteIndex){
  
  char defaultString[] = "Error: Could not retrieve quote."; 
  char* quote = (char*)malloc(sizeof(char)*800); 
  strcpy(quote, ""); 
  int preQuoteIndex = quoteIndex*2; //In the text, 2 endlines flag the quote they precede
  
  File file1; 
  file1 = SD.open(txtFile);  //Open file for reading 
  
  if(file1){
    #if DEBUG
    while (file1.available()){
      Serial.write(file1.read());
    }
    #endif
    
    //Formula: (quoteIndex*2)-2 gives the number of endlines to check for before actually storing what is read
    int i=0; //Use by read() to find where in the text file we should begin reading the quote
    while((i <= preQuoteIndex-2) && (i != preQuoteIndex-2)){  //Move the reader cursor to the start of the quote
      if(file1.read() == '\n')
        i++; 
    }
    
    //Store the quote for return
    strcpy(quote, ""); //Clear the string
    char newLetter[2]; 
    newLetter[1] = '\0'; 
    while(file1.peek() != '\n'){
      newLetter[0] = file1.read(); //Generate the character string
      strcat(quote, newLetter); //Append the next letter
    }
    Serial.println(quote); //DEBUG
    free(quote);  
    file1.close(); 
  }else{ Serial.println("Error opening .txt for reading.");}
   
  return quote; 
}


void setup() {
  Serial.begin(9600); 
  
  //INITIATE SD
  //pinMode(10, OUTPUT); //SPI Requirement -- NOT NEEDED FOR SOME REASON?INCLUSION CAUSES CRASH
  if(SD.begin(4) == true){Serial.println("SD init success"); Serial.println();}
    else if(SD.begin(4) == false){Serial.println("SD init failure"); Serial.println();}

  //Call getQuote("U_Quotes.txt", index) to get a quote for display 
    getQuote("U_Quotes.txt", 1); 
    getQuote("U_Quotes.txt", 2); //Comment this out or program fails!
    //getQuote("U_Quotes.txt", 3); 
}

void loop() {
  
}

U_QUOTES.txt (5.3 KB)

char* getString(char file[], int i);

This seems very dubious, a file is not a char array.

michinyon:

char* getString(char file[], int i);

This seems very dubious, a file is not a char array.

The name of the file is, though.

OP, you MUST make sure that you free the space that you allocate, even of you can't open the file. Or, you should not allocate the memory if you can't open the file.

I worked under the assumption that after closing a file, reopenning the same file causes the byte pointer (from which read() pulls a byte) to point to the very beginning (and not point at the last thing that it pointed to before closing.

This is a correct assumption.

    char newLetter[2]; 
    newLetter[1] = '\0'; 
    while(file1.peek() != '\n'){
      newLetter[0] = file1.read(); //Generate the character string
      strcat(quote, newLetter); //Append the next letter
    }

This, on the other hand, is silly. Use an index variable, starting at 0, incremented each time you store data in quote.

Are the two calls to getString() (which really should be called getQuote()), separated in time? The file.close() function may take some time to get everything done, but it is not a blocking function, so the file may not be read to read again immediately.

OP, you MUST make sure that you free the space that you allocate, even of you can't open the file. Or, you should not allocate the memory if you can't open the file.
This is a correct assumption.

That seems to have fixed the issue. I changed the scope of the malloc call so that it would only be called if file.open() was successful. In the same if statement, I made sure to call free().

void getQuote(char txtFile[], int quoteIndex){
  
  int preQuoteIndex = quoteIndex*2; //In the text, 2 endlines flag the quote they follow
  
  File file1; 
  file1 = SD.open(txtFile);  //Open file for reading 
  
  if(file1){
    char* quote = (char*)malloc(sizeof(char)*800);  

    int i=0; //Formula: (quoteIndex*2)-2 gives the number of endlines to check for before actually storing what is read
    while((i <= preQuoteIndex-2) && (i != preQuoteIndex-2)){  //Move the reader cursor to the start of the quote
      if(file1.read() == '\n')
        i++; 
    }
    
    //Store the quote for return
    strcpy(quote, ""); //Clear the string
    char newLetter[2]; 
    newLetter[1] = '\0'; 
    while(file1.peek() != '\n'){
      newLetter[0] = file1.read(); //Generate the character string
      strcat(quote, newLetter); //Append the next letter
    }
    Serial.println(quote); //DEBUG
    free(quote);
    file1.close(); 
  }else{ 
    Serial.println("Error opening .txt for reading.");
  }
  
}


void setup() {
  Serial.begin(9600); 
  
  //INITIATE SD
  //pinMode(10, OUTPUT); //SPI Requirement -- NOT NEEDED FOR SOME REASON?INCLUSION CAUSES CRASH
  if(SD.begin(4) == true){Serial.println("SD init success"); Serial.println();}
    else if(SD.begin(4) == false){Serial.println("SD init failure"); Serial.println();}

  //Call getQuote("U_Quotes.txt", index) to get a quote for display 
  int i;
  for(i=1;i<31;i++)
    getQuote("U_Quotes.txt", 15); 

}

void loop() {
  
}

This, on the other hand, is silly. Use an index variable, starting at 0, incremented each time you store data in quote.

I just needed a quick, hacky, way to concatenate a char to a string and strcat() operates on two string. I don't know how keeping a counter would help me do concatenation.

Are the two calls to getString() (which really should be called getQuote()), separated in time? The file.close() function may take some time to get everything done, but it is not a blocking function, so the file may not be read to read again immediately.

Perhaps, but this didn't seem to be an issue. I tested this by running getQuote() one right after the other with different quote index after running the fix you suggested.

What memory is used for malloc() calls?

Encapsulation may not be possible if I cannot make multiple malloc() calls to copy the string obtained from the .txt file to the one that needs to be return (the function returns a char*). This would require twice as many bytes as needed to store a quote, which might not be available if a quote is very long.

I'm afraid this is a memory constraint issue.

You could read the file twice. The first time you'd record the position where the quote of interest started, and how long the quote is. Then, you'd malloc() just enough space to hold the quote. Return the pointer to that memory, and let the caller free it when done.

On the second pass, you'd be able to jump right to the start of the quote, and know how many characters to read.

char* quote = (char*)malloc(sizeof(char)*800);

On an Uno that's 40% of your RAM chomped in one go...

MarkT:

char* quote = (char*)malloc(sizeof(char)*800);

On an Uno that's 40% of your RAM chomped in one go...

I need to obtain a string and feed it through an LED matrix (which can only display a certain number of characters at a time).

One way to do this involves retrieving the quote from the U_Quotes.txt file, store a copy of the quote on arduino's SRAM, break down that string into smaller strings(using a little more SRAM), display each of the strings, and then free the memory that was used. Since I have only about 2kb ... this seems a bit risky.

Does this mean that while this process goes on, there isn't much memory for me to do work that involves storing stuff into SRAM?

It might be better to write and read the smaller strings into the SD card but I'm afraid this might introduce delays. What do you think?

One way to do this involves retrieving the quote from the U_Quotes.txt file, store a copy of the quote on arduino's SRAM, break down that string into smaller strings(using a little more SRAM)

Why do you need to copy the data to another location in memory to display it? Why can't you just say "show the characters from n to m, relative to this address (where the address is that of the array and n and m are indexes into that array)?

What is the longest quote that will be displayed on the LED matrix? How many characters at a time can be displayed? If you are only displaying a few, does trying to remember 800 characters make sense? Once they've scrolled by, people would be expected to remember them in order to understand what was scrolled past.

800 characters is a LOT of data.

(My whole response was only 596 characters.)

PaulS:
Why do you need to copy the data to another location in memory to display it? Why can't you just say "show the characters from n to m, relative to this address (where the address is that of the array and n and m are indexes into that array)?

What is the longest quote that will be displayed on the LED matrix? How many characters at a time can be displayed? If you are only displaying a few, does trying to remember 800 characters make sense? Once they've scrolled by, people would be expected to remember them in order to understand what was scrolled past.

800 characters is a LOT of data.

(My whole response was only 596 characters.)

The device includes a Real Time Clock module and I wanted to display a quote (repeatedly) for 24 hours before moving onto the next quote. Also, the quotes display doubles up as a clock and general purpose text display.

Currently, it can hold up to 8 characters (2-4 words) at any instant (but I plan to extend the display).
The longest quote is 767 characters (with spaces)/143 words. Roosevelt had a lot to say about perseverance.

I'm using the Parola library to display text on 8, 8x8 LED matrices. One of the basic functions that displays text takes in a char[] of the string (terminated by \0) I wish to display.

I think I should leverage the size of the SD card and modify the program to store a quote from U_Quote.txt (in pieces that could be read by Parola) in another text file. This file would then be read from and stored into a temporary char[] which could then be used for display. The temporary char[] is overwritten as needed to reload a different piece of the same quote. Since we're writing from one text file to another, SRAM wouldn't be used.

You might consider putting each quote in a separate file, with easily formed names. Then, randomly choose a quote number, convert the number to a string, and access the file based on the string contents. For instance, if the files are named Q1.txt, Q2.txt, ..., Q1200.txt, you could use

int qNum = random(0,1201);
char quoteName[12];

sprintf(quoteName, "Q%d.txt", qNum);

If the random() function returned 53, then quoteName would contain "Q53.txt", which is exactly what is needed to open a file with that name.

Then, break the file up into records, where each record is the amount that can be handled by the display function. That way, there is no reason to read the whole file at once.