Another read text filed from SD card thread...

Hey gang--

I have searched and read a few threads/posts on how to read a text file from an SD card..

no demos/threads were exactly like mine set-up.. so I have a few questions.

summary:

I want to have a .txt file on the SD card that has a few variables/parameters in it.. that are loaded/used in the rest of my sketch

1.)I believe I read that having a start & ending character (delimiter) is best? to differentiate between different values? Is this true? (I mostly read about that for parsing serial data?..but I have no experience either either yet)

2.) to help make sense of the data IN the text field..

Id like to do something along the lines of this:

sOn=0
aCount=99
mColor=r
aColor=b

Or even this would be acceptable (going back to a delimiting character)

<sOn=0>
<aCount=99>
<mColor=r>
<aColor=b>

if that makes it easier?

Is there a way to use the 'name' of the value/variable in the text file is parsing?
or do you just have to KNOW the order of the variables in the text file?

and hence a name wouldnt even be used?.. just:

0
99
r
b
??

Which is NOT my first choice..

for example.. in my sketch I have a var declared called: maxAmmo

but it is HARD CODED in the sketch..

Id liek to eventually load/read/parse the text file..

find the aCount value and assign it the maxAmmo var in my sketch..

make sense?

Im looking for more info/links/discussion about the parsing aspect...
and how to make use of the variabel NAMES in the text file along with assign thosevalues to vars in my sketch..etc.

thanks

Id like to do something along the lines of this:

sOn=0
aCount=99
mColor=r
aColor=b

Or even this would be acceptable (going back to a delimiting character)

<sOn=0>
<aCount=99>
<mColor=r>
<aColor=b>

if that makes it easier?

In both cases, you have an implied end-of-record marker (the carriage return). Adding another one doesn't really buy you anything.

Is there a way to use the 'name' of the value/variable in the text file is parsing?
or do you just have to KNOW the order of the variables in the text file?

Of course there is:

if(strcmp(nameToken, "sOn") == 0)
   sOn = atoi(valueToken); // Where sOn is a variable defined in your sketch

for example.. in my sketch I have a var declared called: maxAmmo

Then, one of the records in the file should be

maxAmmo = someValue

not

aCount = someValue

At least, that's the way I'd do it.

Im looking for more info/links/discussion about the parsing aspect...

Google strtok() (not strtok_r() - that's the thread-safe version; overkill on a single threaded system).

Thank you PaulS-

that was great info for me, appreiate the reply. :slight_smile:

I wasnt thinking about the carriage return (makes it easier to not add any other text/characters)

(does reading a text file suffer from the same 'problems' as reading serial data? In the sense that you can miss/drop data?.. or not something I need to be concerned about?)

Your suggesting to use the SAME values in my sketch as the ones I name/use in the text file? (got it!)

thanks for the hint/tip of: strtok()

I will google it.. as well as look at a C+ book I have lying around once I get home.

I think I should also look up:

strcmp()

and

atoi()

as those are also new to me..

again thanks for the reply and direct advice! :slight_smile:

(does reading a text file suffer from the same 'problems' as reading serial data? In the sense that you can miss/drop data?.. or not something I need to be concerned about?)

No. The conduit for data is much shorter, so the potential for interference is much less. Error checking is much easier, too, since you can always re-read the file. There is no buffer to overflow, either.

hi..

finally home from work.. and giving this a shot..

I think Im missing a few of the 'bigger pieces' on how (generally) this is to be handled.. (sorry)..

it seems you have to read in the text file byte by byte (individual characters).....

(similar to serial stuff, from what I have read)

I got this snippet of code from fat16lib author (since I am trying to add this into a hacked up version of WaveHC demo sketch)..

on reading the text file..

//parse data in text file on SD card
  if (!params.open(root, "defaults.txt")) {
    // handle open failure here   
    putstring_nl("open SAMPLE.TXT");
  }
  // read and print file to serial port
  uint8_t c;
  while(params.read(&c, 1) == 1) {
    Serial.print(c);
    //delay(250);
  }
  params.close();

works fine as far outputting it to the serial monitor..

safety=0
maxAmmo=10
mColor=b
aColor=r

is what I see.. and also what I have in the text file..

I had to throw in a temp delay() to test that the data does in fact come over 1 by 1.

because of this.. am I correct on thinking I need to first split the data up by line first (ie: using the carriage return character... is that /r or /n ??)

and THEN do a string compare?

would this be the correct/acceptable approach for this:

create a 'temp' var to hold the individual string data that is coming over.. keep += the next byte/character to this temp var.. until I read a carriage return..

when I reach/encounter a carriage return, dump this temp var I just collected/created to an Array...

clear/empty temp var.. start collecting characters/bytes for the next 'name/value' pair all over again.. until next carriage return is found.. rinse, repeat, until I somehow check for end of file? or more so.. no more data? (there is only 4 lines.. I can even hard code a something for only 4 lines perhaps?)

the C/C+ route confuses me with all the byte talk and very 'low level' (to me) handling we need to do..

I am used to doing:

var someArray:Array = new Array("whatever", "string", "here");

or even mixed data:
var someArray:Array = new Array("whatever", 6, "here");

or even objects inside of the arrays:

var someArray:Array = new Array({name:value, eyes:green}, {name:value2, eyes:blue});

never having to define lengths.. (just as read only attribute to get array length..etc)

or just pushing/popping data into/out of the stack..etc

someArray.push("whatever");

someArray[4] = "new string in index 4 of the array, not 4 characters/bytes";

So any guidance on ditching my old habits for a CLEAR understanding of how to dynamically create/populate arrays..etc..
Do you always have to be sorta 'generous' in your array 'size' to cover anything you MIGHT add to it?

maybe my approach is wrong or over thought?

I tried to check each character as it comes over:

but am getting am error that says:

ISO C++ forbids comparring a pointer to an integer..etc.

//parse data in text file on SD card
  if (!params.open(root, "defaults.txt")) {
    // handle open failure here   
    putstring_nl("open SAMPLE.TXT");
  }
  // read and print file to serial port
  uint8_t c;
  while(params.read(&c, 1) == 1) {
    if(c != "\n"){
      Serial.print(c);
    }else{
      delay(1000);
    }
  }
  params.close();

(seems to be easier if the string is declared in the sketch already!) lol

thanks PaulS.

This is the code to read and print a character at a time.

  uint8_t c;
  while(params.read(&c, 1) == 1) {
    Serial.print(c);
    //delay(250);
  }

If you look at the documentation for read, you'll notice that you can specify an array as the first argument, and that the second argument is the array length (the number of characters to read). So, you can read more than one character at a time.

That would help if all your records were fixed length. You could read an entire record at once.

Since yours are not, you need to store the characters read from the file.

  char record[80]; // store the record here
  byte index = 0;
  uint8_t c;
  while(params.read(&c, 1) == 1)
  {
     if(c == '\n') // If we found the carriage return...
     {
        // parsing happens here, if index > 0

        // reset to read next record
        index = 0;
        record[index] = '\0';
     }
     else if(c != '\r') // ignore line feed
     {
        // save the character
        record[index++] = c;
        // append a NULL
        record[index] = '\0';
     }
  }

Where the comment says parsing happens here, you'd have some code like:

if(index > 0)
{
   char *nameToken = strtok(record, "=");
   if(nameToken)
   {
      char *valueToken = strtok(NULL, "\0");
      if(valueToken)
      {
      }
   }
}

In the (empty) if block, you'd add code to see if the nameToken matched (using strcmp()) any expected values. If so, you could use atoi(), atof(), or strdup() to convert/copy the value to the appropriate type (float, int, or string).

Thanks PaulS-

Im at work and brought my stuff.. (so I can play/test a bit while Im learning)

Before I dive in.. I want to make sure I am correct when you said look at the documentation...

You meant look in .cpp file? (more so, the FatReader.cpp file) correct?

I found this:

//------------------------------------------------------------------------------
/**
 * Read data from a file at starting at the current read position.
 * 
 * \param[out] buf Pointer to the location that will receive the data.
 * 
 * \param[in] count Maximum number of bytes to read.
 * 
 * \return For success read() returns the number of bytes read.
 * A value less than \a count, including zero, will be returned
 * if end of file is reached. 
 * If an error occurs, read() returns -1.  Possible errors include
 * read() called before a file has been opened, corrupt file system
 * or an I/O error occurred.
 */ 
int16_t FatReader::read(void *buf, uint16_t count) {
  uint8_t *dst = (uint8_t *)buf;
  uint16_t nr = 0;
  int16_t n = 0;
  while (nr < count && (n = readBlockData(dst, count - nr)) > 0) {
    if (!seekCur(n)) return -1;
    dst += n;
    nr += n;
  }
  return n < 0 ? -1 : nr;
}

the param[in] param[out] stuff is a bit confusing....but the BUF (first param) says its a pointer to the location that will receive the data... is there where you got it can be an array type of 'location'?

Im going to dive into the code sample(s).. and see whats going on. (Im sure I'll be back)

thanks

the param[in] param[out] stuff is a bit confusing

The in and out designation simply defines whether the variable is an input to or an output from the function.

The intended use of the param[] stuff is by doxygen, not by us mere mortals.

but the BUF (first param) says its a pointer to the location that will receive the data... is there where you got it can be an array type

Exactly. Pointers and arrays are very closely linked concepts. A pointer is simply an address, as is the array. When a function takes an array, one can pass a pointer, instead. When a function takes a pointer, one can sometimes pass an array instead.

Hi PaulS-

Understood abotu pointers & arrays (makes sense)

here is my current code (basically yoru codw with some prints in there so I can follwo code flow..etc

//parse data in text file on SD card
  if (!params.open(root, "defaults.txt")) {
    // handle open failure here   
    putstring_nl("CANT OPEN DEFAULTS.TXT");
  }
  // read and print file to serial port
  char record[80]; // store the record here
  byte index = 0;
  uint8_t c;
  while(params.read(&c, 1) == 1){
    // read until we found the carriage return... 
    if(c == '\n'){    
      // carriage return encountered, parse the data we collected so far
      if(index > 0){
        char *nameToken = strtok(record, "=");
        putstring("NAME TOKEN: ");
        Serial.println(nameToken);
        
        if(nameToken){          
          char *valueToken = strtok(NULL, "\0");
          putstring("VALUE TOKEN: ");
          Serial.println(valueToken);
          
          if(valueToken){                                    
          }
        }
      }
      // reset to read next record
      index = 0;
      record[index] = '\0';
    }
    // or check if we encounter a line feed too...
    else if(c != '\r'){ // ignore line feed
      // save the character
      record[index++] = c;
      // append a NULL
      record[index] = '\0';
    }
  }
  params.close();

I think I get the basics..

1.) char record[80];
create a character array, able to hold [80] characters..

2.) byte index = 0;
var to be used as our incremntal counter for following code

3.) while(params.read(&c, 1) == 1)

We read the (charactes by character) until it returns '0' (no more data) as assign that character to 'c'

*We then evaluate/check 'c' to se if it IS a carriage return (\n) or is NOT a line feed (\r)..

*if NOT a carriage return OR a line feed... we save/push that character (byte?) into the record character array next index.

*increment index by 1 (++)

*append a null character (terminating character?)

Am I correct to assume that in the text file each line ends like this: \r\n ?? (as in the \r in first? and \n is last so we knwo when to start fresh again?)

4.) if a \n (carriage return) IS encountered.. we know this is the end of the line...
We can 'stop' (so to speak)... and then parse/evaluate the data we have been capturing up until we got to the carriage return...

Heres where Im getting a bit unclear again, but I think Im just over thinking it..

so the sub-code (parsing code), is as follows:

if(index > 0)
{
   char *nameToken = strtok(record, "=");
   if(nameToken)
   {
      char *valueToken = strtok(NULL, "\0");
      if(valueToken)
      {
      }
   }
}

and we only enter this 'sub-routine' is we have encountered a \n (carriage return).. meaning the end of a line..and is ok to start parsing the data collected..

since each time we dumped a character into the record[] character array for holding, we incremented the index variable/counter... this will be over/above (greater than) 0

(if greater than 0, it means there is data/characters in the array to evaluate, if 0..nothing is there, exit)

5.) char *nameToken = strtok(record, "=");

create a variable (nameToken) to store the returned 'string' from the strtok() function.. that takes the current data in record[] and parses it by the delimiter (in this case the = sign)

I get that part.. but the next line is a bit confusing..

6.) if(nameToken){

huh? So if nameToken true?? (as in if there IS something? not empty/null/blank?) Is that what this check is for?

*** what does the * (asterik do for these var names?) ***

7.) char *valueToken = strtok(NULL, "\0");

then we do the same again for the valueToken... although Im not clear on what it is you are using for the parameters in the strotok() function?? NULL as the pointer/reference to parse? and what is the delimiter "\0" for? end of line?

I mean I get the print out on the serial monitor correctly and all:

Ready!
NAME TOKEN: safety
VALUE TOKEN: 0
NAME TOKEN: maxAmmo
VALUE TOKEN: 10
NAME TOKEN: mColor
VALUE TOKEN: b
NAME TOKEN: aColor
VALUE TOKEN: r

I guess Im still fuzzy on how/where I shoudl be doing these conditional checks on the valueTokens?

(right after the strtok() functiosn on both name & value tokens?
( the if(nameToken){} / if(valueToken){}portions?? )

so if in the text file I have: maxAmmo=10 on one line..

I need to check to see if nameToken = maxAmmo.. and then assign its valueToken to a var in my sketch..

another side is, if in the text file I have: aColor=r

I need to not only read/know that Im on that nameToken, but then also evaluate the valeToken (although I guess that could be done elsewhere in the script, as long as I create a variable for values?)

I'll stop here.. I know I can ramble on..

thanks for the help.. Your time is not being wasted, I am trying & learning

update:

I remembered your post about using the compare function.. (Im so used to being able to do things like:

if(varA = "some long string"){
//do whatever
}

so I did that and I think Im moving in the right direction... (it is 'ok' practice' to ten just have a few (4) IF statements to check the nameToken? and then a sub-routine on how to handle the valueToken for each nameToken

Im currently having trouble assigning the valueToken to a var already in the sketch:

if(index > 0){
        char *nameToken = strtok(record, "=");
        putstring("NAME TOKEN: ");
        Serial.println(nameToken);

        if(nameToken){   
          //ammo check
          if(strcmp(nameToken, "maxAmmo") == 0){    
            char *valueToken = strtok(NULL, "\0");
            putstring("VALUE TOKEN: ");
            Serial.println(valueToken);

            if(valueToken){
              maxAmmo = atoi(valueToken);              
            }else{
              //if param is missing, default to 99
              maxAmmo = 99;
          }
        }
        Serial.println(nameToken);
        
        if(nameToken){

It's generally a good idea to check the pointer BEFORE dereferencing it.

Am I correct to assume that in the text file each line ends like this: \r\n ?? (as in the \r in first? and \n is last so we knwo when to start fresh again?)

I never remember the order, but I think that the \n comes first. It's easy enough to test.

We can 'stop' (so to speak)... and then parse/evaluate the data we have been capturing up until we got to the carriage return...

Exactly.

and we only enter this 'sub-routine' is we have encountered a \n (carriage return).. meaning the end of a line..and is ok to start parsing the data collected..

Yes.

since each time we dumped a character into the record[] character array for holding, we incremented the index variable/counter... this will be over/above (greater than) 0

Yes. The real purpose is to deal with a \r following a \n. We don't want to parse a string that contains only a \r.

There is a flaw in the program...

    if(c == '\n'){ 
should be

    if(c == '\n' || c == '\r') {

huh? So if nameToken true?? (as in if there IS something? not empty/null/blank?) Is that what this check is for?

It is testing that the pointer is not NULL.

*** what does the * (asterik do for these var names?) ***

It declares that the variable is a pointer to whatever the type is. char * is a pointer to char(s). int * is a pointer to int(s). float * is a pointer to float(s).

although Im not clear on what it is you are using for the parameters in the strotok() function?? NULL as the pointer/reference to parse? and what is the delimiter "\0" for? end of line?

The NULL in the second call tells strtok() to keep parsing the same string. If you specify the string again, parsing will start at the beginning again.

The "\0" delimiters is an empty string, so strtok will return a pointer to the rest of the string.

hey PaulS-

just wanted to come back and say thanks for the help..

it seemed to work great.. and I am now loading/changing my defaults based on what is on the SD card..

here is what I finally put at the ed of the set-up routine:

//parse data in text file on SD card
  if (!params.open(root, "defaults.txt")) {
    // handle open failure here   
    putstring_nl("CANT OPEN DEFAULTS.TXT");
  }
  // read and print file to serial port
  char record[80]; // store the record here
  byte index = 0;
  uint8_t c;
  while(params.read(&c, 1) == 1){
    // read until we found the carriage return... 
    if(c == '\n' || c == '\r') {     
      // carriage return encountered, parse the data we collected so far
      if(index > 0){
        char *nameToken = strtok(record, "=");
        if(nameToken){ 
          //putstring("CURRENT NAME TOKEN: ");
          //Serial.println(nameToken);  
          //ammo default check
          if(strcmp(nameToken, "maxammo") == 0){    
            char *valueToken = strtok(NULL, "\0"); 
            if(valueToken){
              //putstring("VALUE TOKEN: ");
              //Serial.println(valueToken);
              maxAmmo = atoi(valueToken); 
              ammoCount = maxAmmo;
            }
            else{
              //putstring("DEFAULT VALUE USED: ");
              //if param is missing, default to 99
              maxAmmo = 99;
              ammoCount = maxAmmo;
            }
          }
          //safety default check
          if(strcmp(nameToken, "safety") == 0){    
            char *valueToken = strtok(NULL, "\0"); 
            if(valueToken){
              //putstring("VALUE TOKEN: ");
              //Serial.println(valueToken);
              //safety = atoi(valueToken); 
              safety = atoi(valueToken);
            }
            else{
             // putstring("DEFAULT VALUE USED: ");
              //if param is missing, default to 99
              safety = 0;
            }
          }
          //manual color (mColor) default check
          if(strcmp(nameToken, "scolor") == 0){    
            char *valueToken = strtok(NULL, "\0"); 
            if(valueToken){
              //putstring("MANUAL FIRE COLOR: ");
              //Serial.println(valueToken);
              //check to assign red, green or blue pin
              if(strcmp(valueToken, "r") == 0){
                mColor = ledR;
              }
              else if(strcmp(valueToken, "g") == 0){
                mColor = ledG;
              }
              else if(strcmp(valueToken, "b") == 0){
                mColor = ledB;
              }
              else{
                //default pin assigned if value other than r, g, b, is assigned
                mColor = ledB;
              }
            }
            else{
              //putstring("DEFAULT MANUAL COLOR USED: ");
              //if param is missing, default to blue
              mColor = ledB;
            }
          }
          //auto color (aColor) default check
          if(strcmp(nameToken, "acolor") == 0){    
            char *valueToken = strtok(NULL, "\0"); 
            if(valueToken){
              //putstring("AUTO FIRE COLOR: ");
              //Serial.println(valueToken); 
              //check to assign red, green or blue pin
              if(strcmp(valueToken, "r") == 0){
                aColor = ledR;
              }
              else if(strcmp(valueToken, "g") == 0){
                aColor = ledG;
              }
              else if(strcmp(valueToken, "b") == 0){
                aColor = ledB;
              }
              else{
                //default pin assigned if value other than r, g, b, is assigned
                aColor = ledB;
              }
            }
            else{
              //putstring("DEFAULT MANUAL COLOR USED: ");
              //if param is missing, default to red
              aColor = ledR;
            }
          }
        }
      }      
      // reset index to read next record
      index = 0;
      record[index] = '\0';
    }
    // or check if we encounter a line feed too...
    else if(c != '\r'){ // ignore line feed
      // save the character
      record[index++] = c;
      // append a NULL
      record[index] = '\0';
    }
  }
  params.close();

Im sure eventually I'll come back to work on optimization and better coding practices.. (after getting a working version now)

something like a 'switch case' statement would help? streamline make things more optimized?

Also if your in a 'good mood'.. maybe you can tell me why a putstring_ln() line makes my code work..and when commenting it out.. the motor breaks?

http://arduino.cc/forum/index.php/topic,123117.0.html

(my guts says the toggle is too fast to get the motor 'going enough' before being toggled off...and the putstring_nl() gives just enough room to work still?)

thanks

my guts says the toggle is too fast to get the motor 'going enough' before being toggled off...and the putstring_nl() gives just enough room to work still?

I'd say that that is a reasonable guess. Perhaps a delay(10) in it's place?

*(remembers delay() being the devil?)

this also plays audio..etc..

is just leaving the putstring_ln(); in there bad practice?

(there is no serial 'anything' in my code.. outside of using it for debugging/following code execution)

usually in the end.. I will comment out Serial.begin()..and all putstring/Serial.print lines....etc..
(not clear if that ever helped save space or not?)