Find a string on text file

I've seen numerous examples of this in the forum and implemented the code that is common in the solutions, however, I can't figure out what is wrong with my implementation. I have a text file that has these contents:

ref
ywDheGHD1PVZaky7Ul8T

acc
uBwqUqi6ymXc7N8kayTp

exp
2500

The goal of the function is to update the text file with the token that is being passed to it. This is the function call in the setup:

char kind[] = "ref";
char newToken[] = "Ac6H7pMlAcUxZsLCOmqx";
updateTokens(kind, newToken);

and here is the function:

void updateTokens(char *which, char *theToken) {
  String keepToken = "";
  int tokenPos = 0;
  char buf[5];
  dataFile = sd.open("tokens.txt", FILE_READ);
  if(!dataFile){
    Serial.println(F("File not found"));
    return;
  } else{
    if(strncmp(which, "ref", 3) == 0){
      Serial.println("keeping the access token");
      while(dataFile.available()){
        dataFile.read(buf, 3);
        Serial.print("buffer: "); Serial.println(buf);
        if(strncmp(buf, "acc", 3) == 0){
          tokenPos = dataFile.position() + 1;
          Serial.print("found changing token at pos: "); Serial.println(tokenPos);
          break;
        }
      }
    }
    else if(strncmp(which, "acc", 3) == 0){
      Serial.println("keeping the refresh token");
      while(dataFile.available()){
        dataFile.read(buf, 5);
        if(strncmp(buf, "ref", 3) == 0){
          tokenPos = dataFile.position() + 1;
          Serial.print("found changing token at pos: "); Serial.println(tokenPos);
          break;
        }
      }
    }
    char r;
    dataFile.seek(tokenPos);
    r = dataFile.read();
    keepToken += r;
    keepToken += dataFile.readStringUntil('\r');
    Serial.print("keeper token: "); Serial.println(keepToken);
  }
  
  dataFile.close();
  sd.remove("tokens.txt");
  delay(10);

  Serial.println("time to write the file");
  dataFile = sd.open("tokens.txt", FILE_WRITE);
  if(!dataFile){
    Serial.println(F("File not created"));
    return;
  } else{
    if(strncmp(which, "ref", 3) == 0){
      Serial.println("writing new refresh token");
      dataFile.println("ref");
      dataFile.println(String(theToken));
      dataFile.println();
      dataFile.println("acc");
      dataFile.println(keepToken);
      dataFile.println();
      dataFile.println("exp");
      dataFile.println();
    }
    else if(strncmp(which, "acc", 3) == 0){
      Serial.println("writing new access token");
      dataFile.println("ref");
      dataFile.println(keepToken);
      dataFile.println();
      dataFile.println("acc");
      dataFile.println(String(theToken));
      dataFile.println();
      dataFile.println("exp");
      dataFile.println();
    }
  }
  dataFile.close();
}

And this is the output from the Serial (note there is a character after the ?, it just doesn't show up). I would have anticipated it would go through the file character by character and make a buffer of 5 from that point then eventually find the text and trigger the if statement. Neither of those things seem to be happening.

22:28:38.049 -> keeping the access token
22:28:38.049 -> buffer: ref?
22:28:38.049 -> buffer: 
22:28:38.049 -> y?
22:28:38.049 -> buffer: wDh?
22:28:38.049 -> buffer: eGH?
22:28:38.049 -> buffer: D1P?
22:28:38.049 -> buffer: VZa?
22:28:38.049 -> buffer: ky7?
22:28:38.049 -> buffer: Ul8?
22:28:38.049 -> buffer: T
22:28:38.049 -> ?
22:28:38.049 -> buffer: 
22:28:38.049 -> a?
22:28:38.049 -> buffer: cc?
22:28:38.049 -> buffer: 
22:28:38.049 -> uB?
22:28:38.049 -> buffer: wqU?
22:28:38.049 -> buffer: qi6?
22:28:38.049 -> buffer: ymX?
22:28:38.049 -> buffer: c7N?
22:28:38.049 -> buffer: 8ka?
22:28:38.049 -> buffer: yTp?
22:28:38.049 -> buffer: 
22:28:38.049 -> ?
22:28:38.049 -> buffer: 
22:28:38.049 -> ex?
22:28:38.049 -> buffer: p
22:28:38.049 -> ?
22:28:38.049 -> buffer: 250?
22:28:38.049 -> buffer: 0
22:28:38.049 -> ?
22:28:38.049 -> keeper token: ref

Hello

Are the tokens always the same length (20 chars) ? If yes then you could simply overwrite the token instead of reading and rewriting the complete file

I'm not counting on the tokens always being the same length. But for interest, how would I do as you suggest?

You would open the file with mode O_READ | O_WRITE, which, unlike mode FILE_WRITE, will not append to the end of the file but write at the seeked position

You read the file 3 bytes or 5 bytes at a time and you have variable length tokens… unless you are looking for the first token (direct match at first read) nothing guarantees you’ll be aligned on the keyword when reading.
You should read 3 bytes (if the key is always 3 bytes) and then skip until the new line (CR LF or just CR ?) before attempting to read the next keyword

The ? Is likely because you don’t add a null char after reading the 3 or 5 bytes (you would need 6 bytes in the buffer when you read 5 bytes from the file, space for the trailing null char)

Perhaps I misunderstand file.read() then. I thought this went byte by byte and I create a buffer of 3 or 5 bytes from each byte I go through. So wouldn't I always align eventually and end up finding the key (ref or acc)?

You should read 3 bytes (if the key is always 3 bytes) and then skip until the new line (CR LF or just CR ?) before attempting to read the next keyword

How would I go about this? Would I create a large enough buffer to read a whole line using file.readStringUntil('\r') and test it for the key then move to after the key and load what follows as the token?

this is reading 3 bytes in one go

or 5 bytes here

and then moving the reading head to after that last byte

so when you do

      while (dataFile.available()) {
        dataFile.read(buf, 3);
        Serial.print("buffer: "); Serial.println(buf);
        if (strncmp(buf, "acc", 3) == 0) {
          tokenPos = dataFile.position() + 1;
          Serial.print("found changing token at pos: "); Serial.println(tokenPos);
          break;
        }
      }

you walk through the file 3 bytes by 3 bytes. if you are very lucky the buffer will be exactly aligned and match "acc" but chances are that you'll read "\rac" or "cc " or whatever since the token can be of variable length

you can see that when you walk through

ref
ywDheGHD1PVZaky7Ul8T

you get your output 3 bytes by 3 bytes

22:28:38.049 -> buffer: ref?
22:28:38.049 -> buffer: 
22:28:38.049 -> y?
22:28:38.049 -> buffer: wDh?
22:28:38.049 -> buffer: eGH?
22:28:38.049 -> buffer: D1P?
22:28:38.049 -> buffer: VZa?
22:28:38.049 -> buffer: ky7?
22:28:38.049 -> buffer: Ul8?
22:28:38.049 -> buffer: T

the ? being due to the lack of trailing null which you could fix by doing

int n = dataFile.read(buf, 3); // returns the number of bytes actually read
buf[n] = '\0';

when you read 5 bytes you need one extra byte for the trailing null so define

char buf[6];

Cool. That makes sense to me now. Do I always need to add in the null character before I do a strncmp?

So would something like this work?

while (dataFile.available()) {
    dataFile.read(buf, 3);
    Serial.print("buffer: "); Serial.println(buf);
    if (strncmp(buf, "acc", 3) == 0) {
      tokenPos = dataFile.position() + 1;
      Serial.print("found changing token at pos: "); Serial.println(tokenPos);
      break;
    } else{
      char advance = dataFile.readStringUntil('\n');   //move to the end of that line
    }
}

Or is there a better way?

sounds better as long as you know there is always an \n at the end of each line, including the last one

advance is useless here, just don't store the result

you need to add the trailing null char at the end of buf if you want to print it (which you do) but if you just use strncmp() then you are fine

Right. I guess I could have just done dataFile.readStringUntil('\n')

These null characters really mess me up. Both in the accounting for them in code and which formats like a txt include which ones.

as mentioned previously, just do

int n = dataFile.read(buf, 3); // returns the number of bytes actually read
buf[n] = '\0';

Beauty. Works as expected. The blank lines in the txt file threw it off but I just deleted them.

great ! have fun

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.