Go Down

Topic: Storing calibration tables on an SD card (Read 4856 times) previous topic - next topic

PaulS

Quote
It is probably interpreted as an italic(?) flag by the forum engine

Exactly. b, u, and s in brackets do things, too.

81Pantah

Okay, I am really trying here. This goes WAY beyond my understanding of programming. I have zero C++ background and it seems like strtok() is a native C function. This means the examples I can find are near impossible to understand. I have managed to get it to do what seems right (after a ton of trial and error) but it isn't right.

Here is what I understand my problems to be:

1)  char buf[6]; is a problem. I don't necessarily know what the length of the calibration value is. Some are like 0.00, some are like ?0.10, and most are like 12.34. I tried char buf[5] but that starting returning gibberish. Basically, I don't want this at all. I want to retrieve just the chunks in between the commas. I can't get that to work.

2) My debug output looks great... but it isn't inserting the right values into the array. It seems like it is putting the last value into every array element.

3) Right now, I am swapping variable types just to get the thing to compile. I want to build an array of floats. To get this to work, I can only get char*s in there. I don't seem to be able to parse the serial data for a comma without using char*s.

Code: [Select]


#include <SdFat.h>
#include <SdFatUtil.h>

Sd2Card card;
SdVolume volume;
SdFile root;
SdFile file;

char* responseArray[] = {};

// store error strings in flash to save RAM
#define error(s) error_P(PSTR(s))

void error_P(const char* str) {
  PgmPrint("error: ");
  SerialPrintln_P(str);
  if (card.errorCode()) {
    PgmPrint("SD error: ");
    Serial.print(card.errorCode(), HEX);
    Serial.print(',');
    Serial.println(card.errorData(), HEX);
  }
  while(1);
}

void setup(void) {
 
  Serial.begin(9600);
  Serial.println();
  Serial.println("Type any character to start");
  while (!Serial.available());
  Serial.println();
 
  // initialize the SD card at SPI_HALF_SPEED to avoid bus errors with
  // breadboards.  use SPI_FULL_SPEED for better performance.
  if (!card.init(SPI_HALF_SPEED)) error("card.init failed");
 
  // initialize a FAT volume
  if (!volume.init(&card)) error("volume.init failed");
 
  // open the root directory
  if (!root.openRoot(&volume)) error("openRoot failed");
 
  // open a file
  if (file.open(&root, "CSV.TXT", O_READ)) {
    Serial.println("Opened CSV.TXT");
  }
 
  else{
    error("file.open failed");
  }
 
  int16_t n;
  char* parameter; 
  int parserIteration = 0;
 
  char buf[6]; // THIS IS BASICALLY HARD-CODING. I CAN'T GUARANTEE I KNOW THE LENGTH OF THE VALUES IN THE TXT FILE.
 
  while ((n = file.read(buf, sizeof(buf))) > 0)
  {
      Serial.print("Entering ");
      parameter = strtok(buf, ",");       
      Serial.print(parameter);
      Serial.print(" into responseArray[");
      Serial.print(parserIteration);
      Serial.println("]");
     
      responseArray[parserIteration] = parameter; // THESE VALUES ARE GOING INTO THE ARRAY AS CHAR* ... I NEED THEM IN THERE AS FLOATS.
     
      parserIteration++;
       
      while (parameter != NULL)
      {
        parameter = strtok(NULL, ","); // NOT SURE WHAT THIS DOES, BUT IT BREAKS IF I DON'T HAVE IT
      }
  }

  Serial.println("\nDone\n");
 
  Serial.println("Now, test the array...\n");
 
  // AND THIS DOESN'T REALLY WORK AT ALL, DESPITE LOOKING CORRECT IN MY DEBUG OUTPUT
 
  Serial.print("responseArray[0] now = ");
  Serial.println(responseArray[0]);
  Serial.print("responseArray[1] now = ");
  Serial.println(responseArray[1]);
  Serial.print("responseArray[18] now = ");
  Serial.println(responseArray[18]);
}

void loop(void) {}


and my seemingly awesome serial output that does nothing...

Code: [Select]
Type any character to start

Opened CSV.TXT
Entering 00.00 into responseArray[0]
Entering 00.08 into responseArray[1]
Entering -0.10 into responseArray[2]
Entering 00.56 into responseArray[3]
Entering 01.36 into responseArray[4]
Entering 01.89 into responseArray[5]
Entering 03.15 into responseArray[6]
Entering 03.15 into responseArray[7]
Entering 05.23 into responseArray[8]
Entering 06.20 into responseArray[9]
Entering 07.34 into responseArray[10]
Entering 09.71 into responseArray[11]
Entering 09.29 into responseArray[12]
Entering 10.76 into responseArray[13]
Entering 13.43 into responseArray[14]
Entering 17.42 into responseArray[15]
Entering 16.76 into responseArray[16]
Entering 21.10 into responseArray[17]
Entering 23.51 into responseArray[18]
Entering 26.18 into responseArray[19]
Entering 27.96 into responseArray[20]
Entering 25.20 into responseArray[21]
Entering 25.83 into responseArray[22]
Entering 25.56 into responseArray[23]
Entering 26.84 into responseArray[24]
Entering 28.69 into responseArray[25]
Entering 27.88 into responseArray[26]
Entering 23.08 into responseArray[27]
Entering 25.11 into responseArray[28]
Entering 25.29 into responseArray[29]
Entering 24.52
*@ into responseArray[30]

Done

Now, test the array...

responseArray[0] now = 24.52
*@
responseArray[1] now =
responseArray[18] now = 24.52
*@



Getting closer!

81Pantah


I liked this approach Rob, particularly because you said, "read from a file until 8 chars are read OR a "," is found" That OR is exactly what I want. My problem is I can't figure out the syntax of what should be in a readUntil() function.




Code: [Select]

float readFloat()
{
  char buf[10];
  readUntil(buf, ',' , 8);  // read from file until 8 chars are read or until a ',' is found
  float rv = atof(buf);
  return rv;
}



81Pantah


You are correct that the problem is not in the reading of the data. The data is read from the card correctly. There is no reason to copy it to another array.


Hi Paul, in rereading your comments, I am realizing that I may be missing the big picture. I thought I had to put the data into an array so I could access it later on. My idea was to read the calibration table from the SD card during setup, and then use it with some form of linear interpolation during the main loop. I only want to read the SD card once, no?

Am I missing a best practice for this type of stuff? Learning a lot, that's for sure.

Ryan

PaulS

Quote
I thought I had to put the data into an array so I could access it later on.

You do. But that is exactly what file.read() is doing. You do not need to copy that array to another array.

What you want to store in another array is the numbers read from the file, not the characters.

You could create a readUntil() function that took an array of delimiters (or a hardcoded set). Make that function read one character at a time from the file. Compare that character to each value in the array of delimiters. If no match, store the character in the output array. If there is a match, return.

You would then call that function with a comma, a carriage return, and a line feed in the array of delimiters. The function would return a string that contained the representation of a single floating point number, which atof() would like.

Be sure to check the length of the string before calling atof(). The carriage return and line feed are adjacent in the file, so with both as delimiters, the function would return an empty string after the first was found, when the second one was. You want to ignore that empty string.

81Pantah

Progress!!!

Thank you everyone for your help so far. I am very close.

I have my code successfully reading the CSV-formatted text file from the SD card, parsing that data, and inserting the floats into my array. Yeah.

Now, I have two strange things happening:

1) My code seems to be inserting the values correctly, but when I read them from the array a second time, the first 3 or so are off. Initially, the values go in as 00.00, 00.08, -0.10, 00.56 correctly. But, when I run some Serial.println()s to check what is in the array, the first 3 or so come out as 0.00. Any idea why that is happening?

2) My function to calculate the length of the responseArray[] seems to have stopped working as well. It returns a length of 0 now. Not sure why that is happening either. This code used to work.

Current code:
Code: [Select]

#include <SdFat.h>
#include <SdFatUtil.h>

Sd2Card card;
SdVolume volume;
SdFile root;
SdFile file;

float responseArray[] = {};
char CSV_SEPARATOR = ',';
int elementIdx = 0;
char code[] = "0000000000";
int csvSeparatorCount;

// store error strings in flash to save RAM
#define error(s) error_P(PSTR(s))

void error_P(const char* str) {
  PgmPrint("error: ");
  SerialPrintln_P(str);
  if (card.errorCode()) {
    PgmPrint("SD error: ");
    Serial.print(card.errorCode(), HEX);
    Serial.print(',');
    Serial.println(card.errorData(), HEX);
  }
  while(1);
}

void setup(void)
{
    Serial.begin(9600);
    Serial.println("\nType any character to start");
    while (!Serial.available());
    Serial.println();
    String tmpString = "";
    boolean equalsReadToken = false;
   
   
    // initialize the SD card at SPI_HALF_SPEED to avoid bus errors with
    // breadboards.  use SPI_FULL_SPEED for better performance.
    if (!card.init(SPI_HALF_SPEED)) error("card.init failed");
   
    // initialize a FAT volume
    if (!volume.init(&card)) error("volume.init failed");
   
    // open the root directory
    if (!root.openRoot(&volume)) error("openRoot failed");
   
    // open a file
    if (file.open(&root, "CSV.TXT", O_READ)) {
      Serial.println("Opened CSV.TXT");
    }
   
    else
    {
      error("file.open failed");
    }
   
    int16_t c;

    while ((c = file.read()) > 0)
    {

      if ((char)c == CSV_SEPARATOR)
      {
        char carray[tmpString.length() + 1]; //determine size of the array
        tmpString.toCharArray(carray, sizeof(carray)); //put readStringinto an array
        float n = atof(carray); //convert the array into an Integer
       
        responseArray[elementIdx] = n;
       
        Serial.print("Retrieved element ");
        Serial.print(elementIdx);
        Serial.print(" with a value of ");
        Serial.println(responseArray[elementIdx]);

        tmpString = "";
        csvSeparatorCount++;
        if (csvSeparatorCount == 2) {
          csvSeparatorCount = 0;
          elementIdx = 0;
        } else {
          elementIdx++;
        }
      } else {
        csvSeparatorCount = 0;
        if (((char)c != '\n')&&((char)c !='\r')) {  //prevent newline chars to end up in tmpString
          tmpString += (char)c;                    //accumulate the chars to eventually hold the contents of one element
        }
      }
    }
    file.close();
   
    int responseArrayLength = ((sizeof(responseArray)/sizeof(char *))/2);
   
    responseArrayLength = 30; //Why isn't the dynamic version working?
   
    Serial.print("\nSuccessfully loaded a reponse array from the SD card that is ");
    Serial.print(responseArrayLength);
    Serial.println(" elements long.\n");
   
    for (int loopIteration = 0; loopIteration < responseArrayLength; loopIteration++)
    {
      Serial.print("responseArray[");
      Serial.print(loopIteration);
      Serial.print("] = ");
      Serial.println(responseArray[loopIteration]);
    }
   
  Serial.println("\nDone.\n");
}

void loop(void) {}


wildbill

This declares an array of zero length:
Code: [Select]

float responseArray[] = {};

Which explains your point two. It also explains point one - you're stepping on memory that's being used for other things and when other code writes on them too, your results are lost. Try:
Code: [Select]
float responseArray[30];


81Pantah

Setting responseArray[30] definitely solves my problem, but what if the values on the SD card change?

I would like to support being able to increase the "accuracy" of the sensor by having a CSV text file on the SD card that is N values long.

Is there a way to dynamically append the array length so I don't have to worry about not knowing how long the array is before reading the TXT file?

wildbill

You can use malloc or its siblings to dynamically allocate memory for your array. Don't forget that the Arduino doesn't have very much of it though. For best results, you will need to know up front how many readings you have, although you can use realloc if you don't. In this situation though, best to either count the entries by pre-reading the file, or make the first element in the file be the number of readings it contains.

Go Up