Help writing a function for SD line read

I would like to write a function for reading lines from a .txt file according to line number.

The function should allow each line to be called upon in the following kind of situation

for (i=0; 1=12; i++){
SetElementText (ElementNumber[i],contentsFromLine[i])
}

So basically some kind of function that creates a char* array or string for a line of txt, the particular line of text is defined by the line number.

I have been messing around for a long time and cant seem to get anything working.
Could anyone possible help out with a few lines of code?
Ive found some examples online, but they dont seem to work how I need them to.

Do all lines have the same length? If so, you can use seek() to position the file pointer at the beginning if the line and read the line.

If not, you can use readBytesUntil() repeatedly keeping a count till the count matches the line number.

How do you know you've reached the end of the line?

-jim lee

Here is what I have been experimenting with:

void readFileLine(fs::FS &fs, const char * path) {
  File file = SD.open(path);
  while (file.available()) {
    list = file.readStringUntil('\n');
    //Serial.println (list);
    recNum++;
    for (int i = 0; i < 12; i++) {
      if (recNum == (i)) {
        Serial.println (i);
        Serial.println (list);
        delay (1000);
      }
    }

  }
}

To be completely honest, I have no idea what I am doing :confused:

Some background on the project.
Its a datalogger that writes a temperature value to one of 12 GUI text elements, one after another, by pressing a button.
The 12 values can then be saved to an SD card, the values are saved each to a new line.

(everything to this point is working well)

Im currently working on the recall function, trying to write each of the 12 lines back to each of the 12 GUI text elements.
I need to somehow create an SD read function that can be used inside of a for loop

The hypothetical code would allow me to write lines to GUI elements in the following way:

for (int i = 0; i < 11; i++) {
          snprintf(logVal, MAX_STR, "%.2f", ReadLine[i]);
          gslc_ElemSetTxtStr(&m_gui, (textfield[i]) , logVal);
}

ReadLine would be the contents of the line, and [ i ] the line number/index
Then in the same loop the GUI element to be written to is also updated by [ i ] (I have an array called textfield containing element references)
This means that the first line in the .txt file is written to the first element in the textfield array, and so on, until all 12 elements have been updated with the values from the .txt file.

unfortunately I havent been able to get it working.

Here is a larger portion of my code to help create a picture of whats goin on inside my program.

int counter (-1);
char   logTemp[5];
char   nullVal[5];
char   logName[MAX_STR + 1];
static char nullnullval[7] = "---.--";
int recNum = 0;
String list;
bool CbBtnCommon(void* pvGui, void *pvElemRef, gslc_teTouch eTouch, int16_t nX, int16_t nY)
{
  // Typecast the parameters to match the GUI and element types
  gslc_tsGui*     pGui     = (gslc_tsGui*)(pvGui);
  gslc_tsElemRef* pElemRef = (gslc_tsElemRef*)(pvElemRef);
  gslc_tsElem*    pElem    = gslc_GetElemFromRef(pGui, pElemRef);
  gslc_tsElemRef* textfield[] = {
    DATA_TEXT_1,
    DATA_TEXT_2,
    DATA_TEXT_3,
    DATA_TEXT_4,
    DATA_TEXT_5,
    DATA_TEXT_6,
    DATA_TEXT_7,
    DATA_TEXT_8,
    DATA_TEXT_9,
    DATA_TEXT_10,
    DATA_TEXT_11,
    DATA_TEXT_12,
  };
  TS_NTC_103 sensor (0.11, 10);
  float Temp (sensor.getTemp(analogRead(36)));
  float readLog;
  if ( eTouch == GSLC_TOUCH_UP_IN ) {
    // From the element's ID we can determine which button was pressed.
    switch (pElem->nId) {
      case NEXT_DATA_BTN:
        counter++;
        if (counter > 11) {
          counter = 0;
        }
        snprintf(logTemp, MAX_STR, "%.2f", Temp);
        gslc_ElemSetTxtStr(&m_gui, (textfield[counter]) , logTemp);
        break;
      case PREV_DATA_BTN:
        if (counter >= 0)
        {
          snprintf(nullVal, MAX_STR, "%.2f", nullnullval);
          gslc_ElemSetTxtStr(&m_gui, (textfield[counter]) , nullnullval);
          counter --;
        }
        break;
      case RESET_DATA_BTN:
        gslc_SetPageCur(&m_gui, Log_Delete);
        break;
      case SAVE_SD_BTN:
        gslc_SetPageCur(&m_gui, E_PG_POPUP3);
        break;
      case LOG_DEL_Y:
        for (int i = 0; i < 12; i++) {
          snprintf(nullVal, MAX_STR, "%.2f", nullnullval);
          gslc_ElemSetTxtStr(&m_gui, (textfield[i]) , nullnullval);
          delay (10);
        }
        counter = -1;
        gslc_SetPageCur(&m_gui, E_PG5);
        break;
      case LOG_DEL_N:
        gslc_SetPageCur(&m_gui, E_PG5);
        break;
      case OPEN_LOG_BTN:
        readFileLine(SD, logName);
        Serial.println (recNum);
        break;
      case SAVE_LOG_BTN:
        deleteFile(SD, logName);
        writeFile (SD, logName, gslc_ElemGetTxtStr(&m_gui, (textfield[0])));
        appendFile(SD, logName, "\n");
        for (int i = 1; i < 12; i++) {
          appendFile(SD, logName, gslc_ElemGetTxtStr(&m_gui, (textfield[i])));
          appendFile(SD, logName, "\n");
          delay(20);
        }
        break;
      case CLOSE_LOG_PUP:
        gslc_SetPageCur(&m_gui, E_PG5);
        break;

      //<Button Enums !End!>
      default:
        break;
    }
  }
  return true;
}
bool CbListbox(void* pvGui, void* pvElemRef, int16_t nSelId)
{
  gslc_tsGui*     pGui     = (gslc_tsGui*)(pvGui);
  gslc_tsElemRef* pElemRef = (gslc_tsElemRef*)(pvElemRef);
  gslc_tsElem*    pElem    = gslc_GetElemFromRef(pGui, pElemRef);

  if (pElemRef == NULL) {
    return false;
  }

  // From the element's ID we can determine which listbox was active.
  switch (pElem->nId) {
    //<Listbox Enums !Start!>
    case E_ELEM_LISTBOX1:
      if (nSelId != XLISTBOX_SEL_NONE) {
        gslc_ElemXListboxGetItem(&m_gui, pElemRef, nSelId, logName, MAX_STR);
      }
      break;

    //<Listbox Enums !End!>
    default:
      break;
  }
  return true;
}
void readFileLine(fs::FS &fs, const char * path) {
  File file = SD.open(path);
  while (file.available()) {
    list = file.readStringUntil('\n');
    //Serial.println (list);
    recNum++;
    for (int i = 0; i < 12; i++) {
      if (recNum == (i)) {
        Serial.println (i);
        Serial.println (list);
        delay (1000);
      }
    }

  }
}


void writeFile(fs::FS &fs, const char * path, const char * message) {

  File file = fs.open(path, FILE_WRITE);
  file.print(message);
  file.close();
}
void appendFile(fs::FS &fs, const char * path, const char * message) {

  File file = fs.open(path, FILE_APPEND);
  file.print(message);
  file.close();
}
void deleteFile(fs::FS &fs, const char * path) {

  fs.remove(path);
}
void readFile(fs::FS &fs, const char * path) {
  Serial.printf("Reading file: %s\n", path);

  File file = fs.open(path);
  if (!file) {
    Serial.println("Failed to open file for reading");
    return;
  }

  Serial.print("Read from file: ");
  while (file.available()) {
    Serial.write(file.read());
  }
  file.close();
}

Some notes:
-the values are not all the same, its a 2 point float that can range from -40.00 to 150.00
-I was thinking i could determine the end of a line by "\n"
-if its more practical, I could also use .csv. I am by no means bound to .txt files.
-Sorry if the code is a bit rough, been slugging away trying to get everything working, ill make it look pretty later.

Anyone have any tips or hints?

What's a "GUI text element"?

-jim lee

I am using the GUIslice library and builder to produce a kind of wire frame GUI (graphical user interface) for my ESP32 with 2.8TFT.
The GUI has different elements you can use or add, things like buttons, sliders, tick boxes, toggle buttons and so on.
A text element is just as the name describes, its an element containing text.

My data logger is built around the GUI tools and elements from the GUISlice lib, and works like this;
-when a button is pressed, a 2 point float (representing temperature) is applied to a text element (DATA_TEXT_1), and therefore shown on screen.
-when the button is pressed again, the same thing happens, but this time the temperature is applied to the next text element (DATA_TEXT_2)
-the user repeats the process until all 12 readings are taken. (if a mistake is made the user can press the previous buttun to delete the previous reading, and retake it)
-once all 12 readings are taken, the user presses the save button and can select from a number of log names to save the data to (example: logs 1-10 named /Log_1.txt to /Log_10.txt)
-Each of the 12 recorded temperatures is saved to a new line in the selected .txt file, seperated by "\n" (the file is first deleted, then recreated to avoid multiple data sets being saved in the same log).

Everything to this point is working perfectly.

I am now stuck. I am trying to find a way to recall data from a log.
If the user opens a log by selecting the log band pressing the open button, the values saved in the log are applied to the 12 text elements in the GUI.
each line applied to each text element (line 1 applied to DATA_TEXT_1 and so on).

Some info about the text is written to an Element:

void gslc_ElemSetTxtStr	(	gslc_tsGui * 	pGui,gslc_tsElemRef * 	pElemRef,const char * 	pStr)
		
Update the text string associated with an Element.

Parameters
[in]	pGui	Pointer to GUI
[in]	pElemRef	Pointer to Element reference
[in]	pStr	String to copy into element
Returns
none

pGui ia always the same.
PelemRef for the 12 text fields is taken from an array called textfield.
Somehow I need to make pStr the contents of a line by calling the log name and line number.

I hope this helps

More info can be found here:

Any advice or tips? A hint or a link? I'd be very greatful

void readFileLine(const char * path, const int lineNum, char * list) {
  File file = SD.open(path);
  RecNum = 0;
  while (file.available()) {
    list = file.readStringUntil('\n');
    //Serial.println (list);
    recNum++;
    If (recNum = lineNum){
       for (int i = 0; i < 12; i++) {
           if (recNum == (i)) {
           Serial.println (list);
           file.close();
           }
        }
        delay (1000);
      }
    }
  }

I think the above is along the lines you require, though it is untested (even with the compiler) - there is no error checking whatsoever, so if the line number requested does not exist the file is probbly left open. The function should return the line contents in list - so list needs to be large enough to hold a whole line. The Delay servers no purpose other than to make sure the serial output will remain for a short while so you can read it, once debugged the delay should go. I'll leave commenting the code to you, that way it makes sure you understand it (post the commented version back for checking).
This thread should probably be part of your other thread, but I'll leave you to request the mods merge the two, or a mod might do it anyway.

Im not quite sure how to implement it.

When I press the open button a for loop starts that will change the element reference each cycle to the next reference stored in the array.

In the same for loop I also need the next line in the txt file to be read each cycle.

case OPEN_LOG_BTN:
          for (int i = 0; i < 12; i++) {
          gslc_ElemSetTxtStr(&m_gui, (textfield[i]) , readFileLine(logName, [i],list));
          }
break;

This doesnt work, I have no idea if this is how it should be implemented, and it doesnt compile like this.
I have the following error:

\Documents\Arduino\TiretoolNew\TiretoolNew.ino: In lambda function:
TiretoolNew:167:80: error: expected '{' before ',' token
           gslc_ElemSetTxtStr(&m_gui, (textfield[i]) , readFileLine(logName, [i],list));

I can't check the code I've suggested as my PC HD failed and I'm still trying to reload Windoze.
The function I proposed is a void function so I think it should be used thus:

case OPEN_LOG_BTN:
          for (int i = 0; i < 12; i++) {
          readFileLine (logName, i, list);
          gslc_ElemSetTxtStr(&m_gui, (textfield[i]), list);
          }
break;

I think the for loop left in the readFileLine is spurious and just left over, the only part required is the file.close and optional is the print statement.
Not really clear on what the error message is unless it is the rather than i as the passed parameter.

No matter what I do I cant get it working, or even just serial printing the line number I want.

First off I was getting some errors about not being able to convert string to char, I added some extra code for that but its just not working.

void readFileLine(const char * path, const int lineNum, String list) {
  File file = SD.open(path);
  int RecNum = 0;
  Serial.println ("void begin");
  while (file.available()) {
    Serial.print ("while begin");
    String list = file.readStringUntil('\n');

    char list_array [list.length()];
    list.toCharArray(list_array, list.length());
    char* List = list_array;




    //Serial.println (list);
    recNum++;
    if (RecNum = lineNum) {
      Serial.println("if begin");


      for (int i = 0; i < 12; i++) {
        //Serial.println ("for 2 begin");
        if (RecNum == (i)) {
          Serial.println (List);
          Serial.println (list);
          file.close();
        }
      }
      delay (100);
    }
  }
}

Even after I weed out all the error codes and get it to compile it just doesnt work.
Its really doing my head in.

The example you give looks great, you definitley understand what it is im trying to do, but ill be dammed i just cant get it up and running.

i just cant get it up and running.

Please always provide detailed problem reports.

char   logName[MAX_STR + 1];
char   openLog[MAX_STR + 1];
char   logTemp[5];

case OPEN_LOG_BTN:
        for (int i = 0; i < 12; i++) {
          readFileLine(logName, i, logTemp);
          snprintf(openLog, MAX_STR, "%.2f", logTemp);
          gslc_ElemSetTxtStr(&m_gui, (textfield[i]) , logTemp);
        }
        break;

void readFileLine(const char * path, const int lineNum, String list) {
  File file = SD.open(path);
  int  RecNum = 0;
  while (file.available()) {
    list = file.readStringUntil('\n');
    //Serial.println (list);
    RecNum++;
    if (recNum = lineNum) {
      for (int i = 0; i < 12; i++) {
        if (RecNum == (i)) {
          Serial.println (list);
          file.close();
        }
      }
      delay (1000);
    }
  }
}

At this point i am pulling the info from SD and printing it to the GUI element, however it just prints the first line 12 times, intead of printing each of the 12 lines once.

(hope you understand what I mean)

Anyone have any Ideas?

IF you have one of these files ready to go. I'd start a fresh sketch and just concentriat on reading the file and making your call.

Basically if it is only 12 lines, just setup an array and plug the numbers in and index the array everty time you read an \n.

It it just 12 numbers though? I'm not sure of the format of the file you are reading. Maybe you posted it? (I'm lazy and have not read through all the replies.) If not, and its not terribly large. Cut and paste the .txt file here in code tags so we can see.

-jimlee

Why don’t you log all the data as a sequence of binary structs (the struct holding all the data you want to save) on the SD card. Then all records have the same size (recordSize ) and going to «line» n si just a seek() to position n*recordSize and a read() of recordSize bytes into your struct.

@jimLee There will always be 12 lines in the file, if no value is saved then ---.-- is used as a place holder.
So for example if the user saved the log without taking any measurements then the file will look like this:

---.--
---.--
---.--
---.--
---.--
---.--
---.--
---.--
---.--
---.--
---.--
---.--

Otherwise each line will contain a number between -40.00 and 150.00
For example:

23.43
43.23
65.89
103.33

and so on

When the save button is pressed, the old log is always erased and a new log created with the same name and the new values to avoid any conflicting data.
Each log will always have 12 lines, no matter what, even if its saved without taking any/all measurements.

@J-M-L I would like something text based, so that the data can also be proccessed in excell by removing the sd card and sticking it in the PC. Something like .txt or .csv would be fine. Im not sure how the binary struct would work in this manner. Could you please explain more?

Saving the structure would be a binary representation so not immediately usable in Excel but very well suited for fast handling in your program.

Nothing prevents you to generate two files on your SD (usually you have Gigabytes available so space should not be a problem) instead of one. When working in excel, use the .csv file, when working on the arduino use the .bin file. They both have the same content, represented in different ways.

Another option to have a fast access to line number #N would be to maintain a side binary file to your text file recording the positions of the start of the lines. every time you log a new line in the cdv file you record the current position before writing it, saving this position as uint32_t into a binary file gives you indexing into files up to 4GB large and calculating where to read the start of line number N is a simple math sizeof(uint32_t)*N (that is 4 x N basically)