How to parse file using LittleFS? (ESP32)

Projects Programming
1

Hi everyone,
not quite familiar with embedded programming, pointers etc. Trying my best during lockdown.

I am trying to use the function readFile from the examples but changing it a bit to "parse" the file and create objects that will run in RAM and access them more quicker(does that even make sense?).
I want to create an object from the text file. Text file was written by writeFile function of the examples on the format:
30charactersmaximum&30charactersmaximum&0&0
I was thinking initially to parse it as string and split it by & (lazy .net framework hobbyist i am ) but I am trying to change my logic now and learn something new.
Im lost in the middle of nowhere.. I need to "construct" something like
MyClass mc("30charactersmaximum","30charactersmaximum",0,1);
from the data of the text file.
im lost here at the moment if you could please help me:

void readFile2(fs::FS &fs, const char * path){
 Serial.printf("Reading file: %s\r\n", path);

    File file = fs.open(path);
    if(!file || file.isDirectory()){
        Serial.println("- failed to open file for reading");
        return;
    }
    int i=0;
    int myvar[100];
    Serial.println("- read from file:");
    while(file.available()){
    
    myvar[i]=file.read();
    
     //Serial.println(myvar[i]); //prints numbers, Decimal (bytes) of the text in the txt
     //Serial.write(myvar[i]); //prints exactly whats i want
    
    if(myvar[i]=='&'){
            //split ??
    }
    i=i+1;
    }   
   
    file.close();
}

working class:

class MyClass {
  private:
    static constexpr size_t stringSize = 20;
    char cryptoid[stringSize];
    char dname[stringSize];
    uint8_t enabled;
    uint8_t type;

  public:
    MyClass(const char* c, const char* d, uint8_t e, uint8_t t) {
      strncpy(cryptoid, c, stringSize);
      cryptoid[stringSize - 1] = '\0';
      strncpy(dname, d, stringSize);
      dname[stringSize - 1] = '\0';
      enabled = e;
      type = t;
    }

    const char *get_cryptoid() {
      return cryptoid;
    }

    const char *get_dname() {
      return dname;
    }

    uint8_t get_enabled() {
      return enabled;
    }

    uint8_t get_type() {
      return type;
    }
};

void setup() {
  Serial.begin(115200);
  delay(1000);
  MyClass dm1("sdlfkjsddtheiroeute", "soupakf", 1, 3);
  Serial.println(dm1.get_cryptoid());
  Serial.println(dm1.get_dname());
  Serial.println(dm1.get_enabled());
  Serial.println(dm1.get_type());
}

void loop() {
}

Your help is very much appreciated

2

chris700:
I am trying to use the function readFile from the examples but changing it a bit to "parse" the file and create objects that will run in RAM and access them more quicker(does that even make sense?).

Honestly, not by itself. What processor? The AVR can't run code from RAM.

3

aarg:
Honestly, not by itself. What processor? The AVR can't run code from RAM.

im so sorry, edited title. ESP32
edited: dont mean to do some fancy stuff with ram etc. just to have an instance of an object so i can access it imediatelly and not start parsing when i need it. i believe it will be running in RAM thats why i wrote to RAM .if its not RAM i dont mind. just to have the instance done

4

Erik_Baas:
I can't read this:

[color=#222222][/color]

const char *get_cryptoid() {
     return cryptoid;
   }

Not sure why it shows like this? I copied pasted it again.

5

I think your making this too complicated and hard..

You have this constructor right?

MyClass(const char* c, const char* d, uint8_t e, uint8_t t) {

How many bytes do we need?

int numBytes = strlen(c)+1 + strlen(d)+1 + 2; // string lengths + 2 For e & t

uint8_t myBuff = malloc(numBytes); // Now we have the RAM. Time to fill it.

strcpy((char*)myBuff,c); // We stuff in the first string.
char* trace; // A temporary pointer..
trace = (char*)(&(myBuff[strlen(c)+1])); // Point it to the byte past the last string.
strcpy(trace,d); // Stuff in the next string.
myBuff[numBytes-2] = e; // Stuff in the e byte.
myBuff[numBytes-1] = t; // Stuff in the t byte.

Now go write this buffer to the SD drive.

To read it back?

char* c;
char* d;

int numChars = strlen((char*)readBuff); // We got a buffer from the SD drive. Find the string length of c.
c = malloc(numChars +1); // Allocate a new c.
strcpy(c,(char*)readBuff); // FIll the new c.
char* trace = &(readBuff[numChars+1]); // Point at the next string.
int traceChars = strlen(trace); // How long is the next string..
d = malloc(traceChars +1); // Allocate a new d.
strcpy(d,trace); // Fill the new d.
uint8_t e = (uint8_t)trace[traceChars]; // Grab the e.
uint8_t t = (uint8_t)trace[traceChars+1]; // Grab the t.

myClass anObj = new myClass(c,d,e,t);
free(c);
free(d);

Will this work? Donno' I just typed it out on the screen here. 99.9% chance there will be compile errors. But it'll hopefully give you an idea of a possible approach.

-jim lee

6

Thanks for your reply however i do not understand anything. since im hobbyist and too noob i would appreciate if you could help me handle it with my way so i can keep my project going even on a not so "correct" way and then i can learn all those intresting stuff that you posted. thank you

7

dont laugh at my code lol, im trying to merge two char arrays with seperator & and i tried the commands of merging etc and i was getting random errors of char * and char and this and the other. Im trying to send this complete array to LittleFS to write the file.(before i was doing everything string and then c.str() to send it)
what worked is my random code:

char lala[100];
      int j=0;
      for(j=0;j<strlen(cryptoid);j++){
        lala[j]=cryptoid[j];
      }
      lala[j]='&';
      j=j+1;
      for(int k=0;k<strlen(dname);k++){
        lala[j]=dname[k];
        j=j+1;
      }
     lala[j]='&';
     j=j+1;
     lala[j]=enabled;
     j=j+1;
     lala[j]='&';
     j=j+1;
     lala[j]=type;
     j=j+1;
     lala[j]='&';
     j=j+1;
     lala[j]=write_file;
     j=j+1; 
     lala[j] = '\0';
Serial.println(lala);

enabled, type and write_file are declared as int . so when i print them in Serial i get rectangulars instead of numbers..
How could i fix that? Easy fix would be to make everything char since char worked for cryptoid and dname but any other fix? thank you

EDIT:
it worked after i added +48 in every integer. for example

lala[j]=write_file+48;

I thought in chars we write bytes, not ascii... isnt that correct?

8

chris700:
I thought in chars we write bytes, not ascii... isnt that correct?

Not really sure what you mean here, but the rectangles were because you tried to print character one (ascii) and character zero. They're not printable characters but when you added 48, that brought them up to character '0' and character '1' that are printable.

9

wildbill:
Not really sure what you mean here, but the rectangles were because you tried to print character one (ascii) and character zero. They're not printable characters but when you added 48, that brought them up to character '0' and character '1' that are printable.

If you could please help me understand some basics so i can continue without flooding here the forum:

  1. We give to LittleFS input char for filename and char for the message to write to the text file
  2. Those chars are not bits bytes (i mean they are not numbers that represent letters) but actual letters. A B C D etc
  3. The text that its written by LittleFS is actual text A B C D that people can read and not 2314 234 443 345 663 345

Dont get me wrong, i always used to use just Strings in .net framework so im really struggling here..

10

I think you're confusing yourself. A byte is 8 bits and can hold whatever you like from 0 to 255. Ascii is just a convention that we will use a byte containing 48 to represent '0' and 65 to mean 'A'.

Serial.print knows when you pass it a char or an array of char that you want the Ascii representation, just as it knows if you pass it an int that you want the number it represents instead.

11

wildbill:
I think you're confusing yourself. A byte is 8 bits and can hold whatever you like from 0 to 255. Ascii is just a convention that we will use a byte containing 48 to represent '0' and 65 to mean 'A'.

Serial.print knows when you pass it a char or an array of char that you want the Ascii representation, just as it knows if you pass it an int that you want the number it represents instead.

When LittleFS is writing something how does it write it?
Also, what shall I put to lala array to give it as input to LittleFS if i want a 0 in char array, 0 or 48 for example?
A char array holds in every position the DEC or the CHAR of this table?

715×488 27.3 KB

12

chris700:
When LittleFS is writing something how does it write it?

It does nothing special, just takes whatever bytes you pass it and writes them to the file system - it doesn't care what they are.

Also, what shall I put to lala array to give it as input to LittleFS if i want a 0 in char array, 0 or 48 for example?

Doesn't matter as long as you remember what you did when it comes time to read it. Since you probably want it to be human readable though, add the 48.

There is a caveat though which I don't know is necessarily applicable to LittleFS. It you're writing a text file character 26 may have a special meaning - end of file, so best avoid the non-printable characters.

13

Why haven't you pursed the solution of writing / reading the entire class in binary that I suggested in Your Other Thread?

Get a pointer to the class instance and the number of bytes to write:

uint8_t *ptr = (uint8_t *)&dm1;
size_t numBytes = sizeof(dm1);

Then, once the file is open for writing:

 file.write(ptr, numBytes);
 file.close();

To read from the file, first open it for reading. Then something like this:

  bool success = false;
  size_t i = 0;
  while (file.available()) {
    *(ptr + i++) = file.read();
    if (i == numBytes) {
      success = true;
      break;
    }
  }
  file.close();
  
  if(success) {
    Serial.println("Read Succeeded");
  } else {
    Serial.println("Read Failed");
  }
14

A char array holds in every position the DEC or the CHAR of this table?

It holds eight bits in every position. By convention, if you print them you will get the Ascii character shown in that table, but there's no magic telling the computer I'm holding Ascii data in this byte, it's just raw data. If you want to check what's in a particular char you can compare it to 'A' or you can compare it to 65. It's just a pattern of bits.

This means that you can treat the data as whatever you want. It's also why C++ has casting.

15

gfvalvo:
Why haven't you pursed the solution of writing / reading the entire class in binary that I suggested in Your Other Thread?

Get a pointer to the class instance and the number of bytes to write:

uint8_t *ptr = (uint8_t *)&dm1;

size_t numBytes = sizeof(dm1);




Then, once the file is open for writing:

file.write(ptr, numBytes);
file.close();




To read from the file, first open it for reading. Then something like this:

bool success = false;
 size_t i = 0;
 uint8_t *p = ptr;
 while (file.available()) {
   *p++ = file.read();
   i++;
   if (i == numBytes) {
     success = true;
     break;
   }
 }
 file.close();

if(success) {
   Serial.println("Read Succeeded");
 } else {
   Serial.println("Read Failed");
 }

i appreciate your help, bit too advanced for me.ill test it when i finally understand what is going on with those pointers etc! much appriaciated your help.

wildbill:
It holds eight bits in every position. By convention, if you print them you will get the Ascii character shown in that table, but there's no magic telling the computer I'm holding Ascii data in this byte, it's just raw data. If you want to check what's in a particular char you can compare it to 'A' or you can compare it to 65. It's just a pattern of bits.

This means that you can treat the data as whatever you want. It's also why C++ has casting.

Ah so pretty much since i was used to strings i used to compare if mystring ="teststring", now i have to create an array of the string that i want(teststring) to compare it with the array. right? or compare character per character with ''?

16

I tested a line:
char mypath[60]="/mydir/" "hohoho"
and it actually prints /mydir/hohoho

I was wondering how to do it(used to use + in Strings)..
My question is, is there any "easy" way without the loops i used before to put a char array inside?
like:
char mypath[60]="/mydir/" cryptoid ".txt"

cyptoid is a char array.
Thank you

17 
strcpy(mypath,"/mydir/");
strcat(mypath, cryptoid);
strcat(mypath, ".txt");
18

wildbill:

strcpy(mypath,"/mydir/");

strcat(mypath, cryptoid);
strcat(mypath, ".txt");

thank you! since they say always end the char array with '\0'?

char mypath[60];
strcpy(mypath,"/mydir/");
strcat(mypath, cryptoid);
strcat(mypath, ".txt");
strcat(mypath, '\0');

when i try this then esp32 crashes.just reboots like crazy. if i remove the last line it works.

19

strcat takes care of the terminator. If you did have to add one for some reason the syntax is:

strcat(mypath, "\0");
20

wildbill:
strcat takes care of the terminator. If you did have to add one for some reason the syntax is:

strcat(mypath, "\0");

so for strcat we use String input ( " " ) but to add stuff to char array we use ' ' . right?