Functional code for the Esp32 from "ec2021"

matospinto · March 13, 2024, 10:44pm

Hello,

Have tried the code from "ec2021" on this link below, but i don't know why it not works on my Esp32

/Creating a third .txt file based on two other .txt files (SOLVED) - #21 by ec2021

sergioarduino ask about it in that post.

Hi Sergio, have you a functional code for this issue that you can share?

Thanks
Best Regards

jremington · March 13, 2024, 10:56pm

After reading the "How to get the best out of this forum" post, follow the directions and post the required details (code, error messages and examples of any output). Forum members will be happy to help.

ec2021 · March 14, 2024, 9:36am

Hi @matospinto ,

as the code works in the Wokwi simulation and - even more relevant - on the ESP32 of the TO it is probably a hardware issue in your case, preferably the wiring.

As @jremington posted a sound investigation would require more information about your specific setup...

ec2021

matospinto · March 14, 2024, 10:50am

Thank you ec2021,

It seams that both works, i can see the lines of the files with serial.print in the while loop

 createIndexList("FileB.txt"); 
 evaluateFile("FileA.txt") ;

But writeResultToFile("FileC.txt"); write nothing, i can not see anything on the Serial monitor.

    for (auto txt : outList){
       tFile.println(txt);
       Serial.println(txt); // nothing
    }

The showList() also not print anything on the Serial monitor

  for (auto idx : indexList){
    Serial.println(idx); // print nothing 
  }

The Hardware, i don't think, it is a Dev pcb Wroom 4MB direct connect to a PC and it running other programs without problems.

I search for the post from @jremington to see if it have a example code but not found. Perhaps i make a mistake, i'm new in this forum.

Appreciate any help anyone can provide

Regards

ec2021 · March 14, 2024, 11:07am

I assume you use this sketch (from Wokwi https://wokwi.com/projects/371869704522593281 )

#include <list>
#include <SD.h>
#define CS_PIN 5

File root;
std::list<String> indexList;
std::list<String> outList;

void setup() {
  Serial.begin(115200);

  Serial.print("Initializing SD card... ");

  if (!SD.begin(CS_PIN)) {
    Serial.println("Card initialization failed!");
    while (true);
  }

  Serial.println("initialization done.\n");
  createIndexList("FileB.txt");
  evaluateFile("FileA.txt");
  writeResultToFile("FileC.txt");
}

void loop() {
  delay(100);
  // nothing happens after setup finishes.
}

void createIndexList(String aFile){
  indexList.clear();
  String aLine = "";
  File textFile = SD.open("/"+aFile);
  if (textFile) {
    Serial.println("Creating Index List from "+aFile);
    while (textFile.available()) {
      char c = textFile.read();
      if (c >= ' ') {
        aLine = aLine + c;
      } 
      if (c == 10){
        checkForIndexList(aLine);
        aLine = "";
      }
    }
    textFile.close();
    if (aLine != "") {
      checkForIndexList(aLine);
    }
  } else {
    Serial.println("error opening "+aFile);
  }
  Serial.println();
  showList();
} 

void checkForIndexList(String line){
      String idx = stripIndex(line);
        if (idx > "") { 
          indexList.push_back(idx);
          }
}

String stripIndex(String Line){
  int c1 = Line.indexOf(',');
  int c2 = Line.indexOf(',', c1+1);
  String index = "";
  if (c1 >= 0 && c2 > c1){
    index = Line.substring(c1+1, c2);
  }
  return index;
}

void showList(){
  Serial.println("===== Index List ========");
  for (auto idx : indexList){
    Serial.println(idx);
  }
  Serial.println("=========================\n");
}  

void evaluateFile(String aFile){
  String aLine = "";
  outList.clear();
  File textFile = SD.open("/"+aFile);
  if (textFile) {
    Serial.println("Evaluating "+aFile);
    Serial.println("\n======= New Entries ========");
    while (textFile.available()) {
      char c = textFile.read();
      if (c >= ' ') {
        aLine = aLine + c;
      } 
      if (c == 10){
          checkForOutList(aLine);
          aLine = "";
        }
     }
    textFile.close();
    checkForOutList(aLine);
  } else {
    Serial.println("error opening "+aFile);
  }
  Serial.println("============================\n"); 
} 

void checkForOutList(String line){
        String idx = stripIndex(line);
        if (idx > "") { 
           if (!indexFound(idx)) {
            Serial.println(idx);
            outList.push_back(line);
          }
       }
}      

void checkIndex(String line){
      String idx = stripIndex(line);
        if (idx > "") { 
          indexList.push_back(idx);
          }
}

boolean indexFound(String aIndex){
  boolean found = false;
  for (auto ix : indexList){
    if (aIndex == ix) found = true;
  }
  return found;
} 

void writeResultToFile(String wFile){
  File tFile = SD.open("/"+wFile, FILE_WRITE);
  if (tFile) {
    Serial.println("Writing to "+wFile);
    Serial.println("\n========= Entries ========");
    for (auto txt : outList){
       tFile.println(txt);
       Serial.println(txt);
    }
    tFile.close();
    Serial.println("============================\n"); 
  } else {
    Serial.println("error opening "+wFile);
  }
}

Additional Textfiles

FileA.txt:

0,B123,USUARIO TESTE,E5
1,X001,SERGIO FRANCA,F1
2,H002,LUCCA DA MATA,F2
3,M789,EMILLY MONTEIRO,F3
4,S111,JOANA FRANCA,EM1

FileB.txt

0,B123,USUARIO TESTE,E5,segunda-feira,31-07-2023,19:58:00
1,X001,SERGIO FRANCA,F1,segunda-feira,31-07-2023,19:58:00
2,H002,LUCCA DA MATA,F2,segunda-feira,31-07-2023,19:58:00

If yes - provided that you have copied the two files FileA.txt and FileB.txt to the SD Card- the expected Serial output is:

Initializing SD card... initialization done.

Creating Index List from FileB.txt

===== Index List ========
B123
X001
H002
=========================

Evaluating FileA.txt

======= New Entries ========
M789
S111
============================

Writing to FileC.txt

========= Entries ========
3,M789,EMILLY MONTEIRO,F3
4,S111,JOANA FRANCA,EM1
============================

Is something different or missing in your Serial output?
If yes, what is it?

matospinto · March 14, 2024, 11:20am

Hi,
I use not SD but SPIFFS and have make the needed changes. Now i test the code on the WOKWI "i never hear about it", thats amazing.

But i think it not support SPIFFS, it show begin error

if (!SPIFFS.begin(true)) {
    Serial.println("Begin SPIFFS error");
    return;
  }

Regards

ec2021 · March 14, 2024, 12:22pm

It obviously makes a big difference if an unchanged sketch does not work on a similar platform or after it had been modified.

In this case post your complete code, please.

That may give a chance to support.

matospinto · March 14, 2024, 12:32pm

Thank you ec2021,

#include <list>
#include "FS.h"
#include "SPIFFS.h"

File root;
std::list<String> indexList;
std::list<String> outList;

void setup() {
  Serial.begin(115200);

  // Initializa SPIFFS
  if(!SPIFFS.begin(true)){
      Serial.println("SPIFFS BEGIN ERROR");
      return;
  }

  Serial.println("initialization done.\n");
  createIndexList("FileB.txt");
  evaluateFile("FileA.txt");
  writeResultToFile("FileC.txt");
}

void loop() {
  delay(100);
  // nothing happens after setup finishes.
}

void createIndexList(String aFile){
  indexList.clear();
  String aLine = "";
  //File textFile = SD.open("/"+aFile);
  File textFile = SPIFFS.open("/"+aFile);
  if (textFile) {
    Serial.println("Creating Index List from "+aFile);
    while (textFile.available()) {
      char c = textFile.read();
      if (c >= ' ') {
        aLine = aLine + c;
      } 
      if (c == 10){
        checkForIndexList(aLine);
        aLine = "";
      }
    }
    textFile.close();
    if (aLine != "") {
      checkForIndexList(aLine);
    }
  } else {
    Serial.println("error opening "+aFile);
  }
  Serial.println();
  showList();
} 

void checkForIndexList(String line){
      String idx = stripIndex(line);
      Serial.println(idx);
        if (idx > "") { 
          indexList.push_back(idx);
        }
}

String stripIndex(String Line){
  int c1 = Line.indexOf(',');
  int c2 = Line.indexOf(',', c1+1);
  String index = "";
  if (c1 >= 0 && c2 > c1){
    index = Line.substring(c1+1, c2);
  }
  return index;
}

void showList(){
  Serial.println("===== Index List ========");
  for (auto idx : indexList){
    Serial.println(idx);
  }
  Serial.println("=========================\n");
}  

void evaluateFile(String aFile){
  String aLine = "";
  outList.clear();
  //File textFile = SD.open("/"+aFile);
  File textFile = SPIFFS.open("/"+aFile);
  if (textFile) {
    Serial.println("Evaluating "+aFile);
    Serial.println("\n======= New Entries ========");    
      while (textFile.available()) {
      char c = textFile.read();
      if (c >= ' ') {
        aLine = aLine + c;
      } 
      if (c == 10){
        checkForIndexList(aLine);
        aLine = "";
      }
    }
    textFile.close();
    checkForIndexList(aLine);
  }
  Serial.println("============================\n"); 
} 

void checkForOutList(String line){
        String idx = stripIndex(line);
        if (idx > "") { 
           if (!indexFound(idx)) {
            Serial.println(idx);
            outList.push_back(line);
          }
       }
}      

void checkIndex(String line){
      String idx = stripIndex(line);
        if (idx > "") { 
          indexList.push_back(idx);
          }
}

boolean indexFound(String aIndex){
  boolean found = false;
  for (auto ix : indexList){
    if (aIndex == ix) found = true;
  }
  return found;
} 

void writeResultToFile(String wFile){
  //File tFile = SD.open("/"+wFile, FILE_WRITE);
  File tFile = SPIFFS.open("/"+wFile, FILE_WRITE);
  if (tFile) {
    Serial.println("Writing to "+wFile);
    Serial.println("\n========= Entries ========");
    for (auto txt : outList){
       tFile.println(txt);
       Serial.println(txt);
    }
    tFile.close();
    Serial.println("============================\n"); 
  } else {
    Serial.println("error opening "+wFile);
  }
}

The content of FileA.txt
PBW OF723 Rue C, 24
PBW OF541 Av. Jo V, 21
PBW OF724 Rue D, 25
PBW OF725 Rue E, 26
PBW OF726 Rue F, 27
PBW OF727 Rue G, 28

The content of FileB.txt
PBW OF723 Rue C, 24
PBW OF541 Av. Jo V, 21
PBW OF724 Rue D, 25
PBW OF726 Rue F, 27
PBW OF727 Rue G, 28

Regards

ec2021 · March 14, 2024, 1:41pm

At first glance your changes seem fine. So what's interesting now is a copy of the serial output when you run the sketch.

The functions where files are opened do have an error message.

You can post the output also in code tags ...

matospinto · March 14, 2024, 2:07pm

I'm sure it reads the FileA.txt and FileB.txt than when i add a serial.println in the while loop of void createIndexList and void evaluateFile i can see the contents of the files

initialization done.

Creating Index List from FileB.txt

===== Index List ========

Evaluating FileA.txt

======= New Entries ========

============================

Writing to FileC.txt

========= Entries ========

Thanks for your support
Regards

ec2021 · March 14, 2024, 2:42pm

It reads but the index list is empty.

In the original data the index is between the first and second comma. Your data have only one comma

Try to put a comma at the end of each entry or change the evaluation of the index position.

See the function

String stripIndex(String Line);

That should work...

Good luck!
ec2021

matospinto · March 14, 2024, 3:05pm

To test, now i change the lines of the FileA.txt and FileB.txt to this format

PBW, OF723, Rue C, 24
PBW, OF726, Rue F, 27
PBW, OF727, Rue G, 28

and the result is this:

initialization done.

Creating Index List from FileB.txt
 OF723
 OF541
 OF724
 OF726
 OF727

===== Index List ========
 OF723
 OF541
 OF724
 OF726
 OF727
=========================

Evaluating FileA.txt

======= New Entries ========
 OF723
 OF541
 OF724
 OF725
 OF726
 OF727
============================

Writing to FileC.txt

========= Entries ========
============================

ec2021 · March 14, 2024, 4:11pm

To doublecheck I have created a separate instance of the SD simulation on Wokwi:

https://wokwi.com/projects/392348809088077825

No change to the original code from

https://wokwi.com/projects/371869704522593281

except that I copied your data to FileA.txt and FileB.txt.

If I start this simulation I get this result:

Initializing SD card... initialization done.

Creating Index List from FileB.txt

===== Index List ========
OF723
OF541
OF724
OF726
OF727
=========================

Evaluating FileA.txt

======= New Entries ========
OF725
============================

Writing to FileC.txt

========= Entries ========
PBW,OF725,Rue E, 26
============================

which is as expected.

Do you have white spaces between the commas and the begin and end of your indices? The index search does not trim the index strings. If you have it in one and not in the other file the indices are not handled as identical ...

No

I found the reason! You have changed the function evaluateFile and coded

 checkForIndexList(aLine);

instead of

 checkForOutList(aLine);

This is a sketch that should do it now:

Repaired Sketch

/*
  Forum: https://forum.arduino.cc/t/functional-code-for-the-esp32-from-ec2021/1235244/12
  Wokwi: https://wokwi.com/projects/392349773369145345

*/

#define USESPIFF

#include <list>
#include "FS.h"

#ifdef USESPIFF
#include "SPIFFS.h"
#else
#include <SD.h>
#define CS_PIN 5
#endif

File root;
std::list<String> indexList;
std::list<String> outList;

void setup() {
  Serial.begin(115200);

#ifdef USESPIFF
  // Initializa SPIFFS
  if (!SPIFFS.begin(true)) {
    Serial.println("SPIFFS BEGIN ERROR");
    return;
  }
#else
  if (!SD.begin(CS_PIN)) {
    Serial.println("Card initialization failed!");
    while (true);
  }
#endif

  Serial.println("initialization done.\n");
  createIndexList("FileB.txt");
  evaluateFile("FileA.txt");
  writeResultToFile("FileC.txt");
}

void loop() {
  delay(100);
  // nothing happens after setup finishes.
}

void createIndexList(String aFile) {
  indexList.clear();
  String aLine = "";

#ifdef USESPIFF
  File textFile = SPIFFS.open("/" + aFile);
#else
  File textFile = SD.open("/" + aFile);
#endif

  if (textFile) {
    Serial.println("Creating Index List from " + aFile);
    while (textFile.available()) {
      char c = textFile.read();
      if (c >= ' ') {
        aLine = aLine + c;
      }
      if (c == 10) {
        checkForIndexList(aLine);
        aLine = "";
      }
    }
    textFile.close();
    if (aLine != "") {
      checkForIndexList(aLine);
    }
  } else {
    Serial.println("error opening " + aFile);
  }
  Serial.println();
  showList();
}

void checkForIndexList(String line) {
  String idx = stripIndex(line);
  Serial.println(idx);
  if (idx > "") {
    indexList.push_back(idx);
  }
}

String stripIndex(String Line) {
  int c1 = Line.indexOf(',');
  int c2 = Line.indexOf(',', c1 + 1);
  String index = "";
  if (c1 >= 0 && c2 > c1) {
    index = Line.substring(c1 + 1, c2);
  }
  return index;
}

void showList() {
  Serial.println("===== Index List ========");
  for (auto idx : indexList) {
    Serial.println(idx);
  }
  Serial.println("=========================\n");
}



void evaluateFile(String aFile) {
  String aLine = "";
  outList.clear();

#ifdef USESPIFF
  File textFile = SPIFFS.open("/" + aFile);
#else
  File textFile = SD.open("/" + aFile);
#endif

  if (textFile) {
    Serial.println("Evaluating " + aFile);
    Serial.println("\n======= New Entries ========");
    while (textFile.available()) {
      char c = textFile.read();
      if (c >= ' ') {
        aLine = aLine + c;
      }
      if (c == 10) {
        //       checkForIndexList(aLine);
        checkForOutList(aLine);
        aLine = "";
      }
    }
    textFile.close();
    //    checkForIndexList(aLine);
    checkForOutList(aLine);
  }
  Serial.println("============================\n");
}


void checkForOutList(String line) {
  String idx = stripIndex(line);
  if (idx > "") {
    if (!indexFound(idx)) {
      Serial.println(idx);
      outList.push_back(line);
    }
  }
}

void checkIndex(String line) {
  String idx = stripIndex(line);
  if (idx > "") {
    indexList.push_back(idx);
  }
}

boolean indexFound(String aIndex) {
  boolean found = false;
  for (auto ix : indexList) {
    if (aIndex == ix) found = true;
  }
  return found;
}

void writeResultToFile(String wFile) {
#ifdef USESPIFF
  File tFile = SPIFFS.open("/" + wFile, FILE_WRITE);
#else
  File tFile = SD.open("/" + wFile, FILE_WRITE);
#endif

  if (tFile) {
    Serial.println("Writing to " + wFile);
    Serial.println("\n========= Entries ========");
    for (auto txt : outList) {
      tFile.println(txt);
      Serial.println(txt);
    }
    tFile.close();
    Serial.println("============================\n");
  } else {
    Serial.println("error opening " + wFile);
  }
}

I left the SD card parts in there as I tested it on Wokwi. with #define USESPIFF the SPIFFS part is implemented else the SD-Part. Feel free to remove the defines for your final version.

ec2021

matospinto · March 14, 2024, 9:05pm

Thank you very much ec2021

You are the best, now it works!

I have only change the text format to this
PBW OF724 Rue F 27
PBW OF725 Rue G 28

Best regards

ec2021 · March 15, 2024, 7:38am

Great.

Please mark the post that includes the solution so that other members can directly see that the thread has been successfully finalized.

If you change the text format e.g. by removing all commas and using white spaces instead be aware to modify the function stripIndex() accordingly.

matospinto · March 15, 2024, 10:05am

Hi, ec2021

To use textline format like this PWB 0G32 Rue H 24 using space instead commas i change in the Void stripIndex(String Line) this two lines to
int c1 = Line.indexOf(' ');
int c2 = Line.indexOf(' ', c1 + 1);

Now it works like i need.
Here a big Thanks to ic2021 for it's great help.

Regards

ec2021 · March 15, 2024, 11:19am

Hi @matospinto ,

that's correct. White spaces are possible delimiters but probably not the best choice. Commas, semicolons or the like make it easier for us humans to see if there is no, one or more of the delimiters in a row ...

However, as long as the first space only appears before and the second behind the index string everything should work fine!

Have fun!
ec2021

matospinto · March 16, 2024, 5:20pm

Hi ec2021,

Something is wrong. I tested it with Esp32 and since it wasn´t correct I also tested it with wokwi, but the result is the same as with Esp32. Is there a line limit for FileA.txt and FileB.txt ?
========= Entries ========
PBW, OF722, Rue, 2
PBW, OF724, Rue, 5
PBW, OF729, Rue, 10
PBW, OF729, Rue, 15

============================

I miss
PBW, OF730, Rue, 11
PBW, OF726, Rue, 12

Have you a explanation?
To test it i have preload the FileA.txt and FileB.txt on

Thank you
Regards

matospinto · March 16, 2024, 6:27pm

Hi, ec2021,

Now i undestand, it's not the entire line that be evaluate but the only the OFxxx
so, OF1234 Rue 2 is the same as OF1234 Rue 14

Sorry for my undestand mistake

Best regards

ec2021 · March 16, 2024, 8:27pm

That's it. An index should/must be something that is unique.

Topic		Replies	Views
Creating a third .txt file based on two other .txt files (SOLVED) General Guidance	23	904	January 28, 2024
ESP32 und SD Card Frage Deutsch	28	412	July 16, 2025
Problems to understand the character variable types Programming	26	251	February 14, 2026
Best Method for Utilizing On-Board Flash? Nano ESP32	22	5712	November 26, 2025
Problems with sd on arduino nano Programming	26	158	June 10, 2025

Functional code for the Esp32 from "ec2021"

===== Index List ========

========= Entries ========

Related topics