Lost in coding: read float values from a SD card

Using Arduino Programming Questions
1

Dear all,

Let me explain my project to let you better know what i want.

I create a datalogger with multiple ds18b20 sensors connected to 5 different pins of the arduino. The number of sensor in each pin in originally unkonw, because it could change. The code i have works perfectly, but now i want to go a little bit deeper, allowing to include offset values for each sensor.

What i have on mind is to save the values of the offset in a txt file in the SD ard already connected to the device. The during the settings, what i want to do is to read the list of offsets in the txt file and (for the moment), store them into arrays (in the future i will save this values into an external eeprom).

The file could have this content:

A32.12;-5.32;6.32;
B0.00;-0.05;
C4.23;1.03;-4.45;
D
E0.34;

each letter correspond to a different pin in the datalogger.Values are random, do to take ccare about them, but the number of sensors in each channel could be different.

I was fightinng about how to read the values and move them to 5 different arrays of floats (one per channel). But i am completly lost. This is the code i was fighting with, based on the SD readwrite example:

#include <SPI.h>
#include <SD.h>

File myFile;
float myOffsets[5] = {Offset1, Offset2, Offset3, Offset4, Offset5};
void setup(){
  Serial.begin(9600);
  while (!Serial) {
    ;
  }
  Serial.print("Initializing SD card...");
  pinMode(10, OUTPUT);
  if (!SD.begin(10)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  myFile = SD.open("offsets.txt");
  if (myFile) {
    byte a = 0;
    byte b = 0;
    byte i = 0;
    byte pos = 0;
    char digit;
    char letters[6];
    Serial.println("offsets.txt:");

    // read from the file until there's nothing else in it:
    while (myFile.available()) {
      digit = myFile.read();
      Serial.write(digit);
      switch(digit){
        case 'A':
          a = 0;
          break;
        case 'B':
          a = 1;
          break;
        case 'C':
          a = 2;
          break;
        case 'D':
          a = 3;
          break;
        case 'E':
          a = 4;
          break;
        case ';':
          myOffsets[a][b] = atof(letters);
          for (byte c=0; c<5;c++){
            letters[c]='';
          }
          b++;
          i = 0;
          break;
        case '\n':
          b = 0;
          break;
        default:
          letters[i] = digit;
          i++;
          break;
      }
    }
    myFile.close();
    Serial.println(Offset1[]);
    Serial.println(Offset2[]);
    Serial.println(Offset3[]);
    Serial.println(Offset4[]);
    Serial.println(Offset5[]);
  } else 
    Serial.println("error opening test.txt");
}

void loop()
{
  // nothing happens after setup
}

The list of errors is this:

Arduino: 1.5.8 (Windows 8), Placa:"Arduino Nano, ATmega328"

ReadSDvalues.ino:52:24: error: empty character constant
ReadSDvalues.ino:5:23: error: 'Offset1' was not declared in this scope
ReadSDvalues.ino:5:32: error: 'Offset2' was not declared in this scope
ReadSDvalues.ino:5:41: error: 'Offset3' was not declared in this scope
ReadSDvalues.ino:5:50: error: 'Offset4' was not declared in this scope
ReadSDvalues.ino:5:59: error: 'Offset5' was not declared in this scope
ReadSDvalues.ino: In function 'void setup()':
ReadSDvalues.ino:50:25: error: invalid types 'float[byte {aka unsigned char}]' for array subscript
ReadSDvalues.ino:67:20: error: 'Offset1' was not declared in this scope
ReadSDvalues.ino:67:28: error: expected primary-expression before ']' token
ReadSDvalues.ino:68:20: error: 'Offset2' was not declared in this scope
ReadSDvalues.ino:68:28: error: expected primary-expression before ']' token
ReadSDvalues.ino:69:20: error: 'Offset3' was not declared in this scope
ReadSDvalues.ino:69:28: error: expected primary-expression before ']' token
ReadSDvalues.ino:70:20: error: 'Offset4' was not declared in this scope
ReadSDvalues.ino:70:28: error: expected primary-expression before ']' token
ReadSDvalues.ino:71:20: error: 'Offset5' was not declared in this scope
ReadSDvalues.ino:71:28: error: expected primary-expression before ']' token
Error de compilación

This report would have more information with
"Show verbose output during compilation"
enabled in File > Preferences.

I know there is a different sd cards library that is able to read values from CSV filesm but i can´t add more libraries because memory limitations. I also saw other similar ases in this forum, but the people used close matrix of X rows and Y columns. I will alway have 5 rows in the file, but the number of columns in each row is undefined until the user introduce the data...

I am also worry because it seems that i am using strings to manage the readed characters, what is not the best option.

Could you give me ideas about how to solve this issue? I will really appreciate it. Thanks in advance!

2

Why don't you go store that directly in embedded eeprom Of your arduino??

If there is a small max number of sensors per pin ? If so you could just use the embedded EEPROM and store an array of the max size

3

You haven't declared Offset1 etc here:

float myOffsets[5] = {Offset1, Offset2, Offset3, Offset4, Offset5};

Presumably it would be reasonable to do:

float myOffsets[5] = {0.0, 0.0, 0.0, 0.0, 0.0};
4

Thanks @J-M-L,

The number of sensors per channel could be high. I use and eeprom to save the ID numbers of each sensor, and the limit is about 100 sensors. Sometimes, only 1 to 15 ould be connected, but i do notknow how the different future uses could imply a higher number of sensors.

Yes, i designed the datalogger with only one eeprom, but i think i will add another one for the offsets. The current code i have read the serial numbers and store them in the eeprom. Then, when required, the serial numbers are readed in order to call to each sensor and measure the temperature. I Suposse than i could do something similar for the offsets, bbut in any ctase i need to read the values from the SD card. Now i am at this point in the problem.

Thanks!e

5

Thanks @MarkT,

Yes, i know the offset arrays are still not declared. The problem is that i do not know its size,because i do not know how many values per line are savedin the file.

I know that the array delaration require the size. Could i declare a size of 1 and increase this value on runtime?
I am lost also here.
Thanks1

6

Why not one extra line at the beginning of the file that tells how many are in each line?

7

Good point @Delta_G!
In fact, i know how many sensors there are in each channel on runtime. Before to try to read the offsets, my code at the setup section search for all the sensors in each channel, and save the result into an array like this:

for (byte a = 0; a < 5; a++) {                                     // For each channel...
      SensorsBus[a].begin();                                           // Initialize temperature sensor bus
      byte sn1 = SensorsBus[a].getDeviceCount();                       // Establish the number of sensors on the temperature bus
      eeprom_write_byte(a, sn1);                                       // Save in EEPROM the number of sensors in the bus
      sensorsnumber[a] = sn1;}

I was thining that may be it could be easier to read the file line by line. Then, split the values thanks to the use of the separator (";"), convert each value into a float,and send it to the eeprom.

In fact, i did something similar when reading data coming from a bluethooth. The pieces of code (modified from someone in the forum or the web) are this ones:

char *item[13];
(...)

// Process the incoming data
void processIncomingByte (const byte inByte) {
  static unsigned int input_pos = 0;
  switch (inByte) {
    case '\n':
      input_line[input_pos] = 0;
      process_data(input_line);
      input_pos = 0;
      break;
    case '\r':   // discard carriage return
      break;
    default:
      if (input_pos < (MAX_INPUT - 1))
        input_line [input_pos++] = inByte;
      break;
   }
}

// Use the incoming data
void process_data (char * data) {
  char *ptr;
  int count = 0;
  ptr = strtok(data, ";");
  while (ptr) {
    item[count++] = ptr;   
    ptr = strtok(NULL, ";");
    //then, convert each item into a float number and to save it at the eeprom.
  }
}

What do you think? Do you think that this could be also valid for the sd ard reading case? How could i adapt it?

8

I'm still unclear of why you need the SD card at all?

9

Because the user could change the array of sensors connected to each channel. Then, by the use of the sd card, the user do not need to upload a new firmware, just only copy the offsets into the sd card, and restart the datalogger. It checks if an "offsets.txt" file exists. If not, the datalogger just only measure the temperatures and save the data into a "data.txt" file into the sd card. If yes, it correct the measurement considering the offsets before to save the data into the sd card.
I want to reduce as much as possible what the user need to do with the data logger. Just only connect the sensors probes to each channel, introduce a sd card, power it, turn it on, and that is. One year latter, the user only need to take off the sd card, and replace for a new one, and that is. If the user does not change the sensors probe, he does not need the offset file. If one of the probes has a problem or the user want to change it, the the offset file could be used, but it is still optional, when the probe was calibrated.
Then, I think this is the easiest way to work. The user do not need to know what is inside the data logger or how it works. Just an sd card is enough. It is also fast and clean, not exposed to the environmental conditions. Originally, I designed the data logger to be used in Antarctica, where it is not easy to work outside, and neither to interact with the data logger by a notebook to upload a new firmware,... open, change the sd card, and close. Less than a minute.

Thanks!

10

madepablo:
Thanks @MarkT,

Yes, i know the offset arrays are still not declared. The problem is that i do not know its size,because i do not know how many values per line are savedin the file.

I know that the array delaration require the size. Could i declare a size of 1 and increase this value on runtime?
I am lost also here.
Thanks1

You then will need to malloc() the array and have a pointer variable.

11

madepablo:
Because the user could change the array of sensors connected to each channel. Then, by the use of the sd card, the user do not need to upload a new firmware, just only copy the offsets into the sd card, and restart the datalogger. It checks if an "offsets.txt" file exists. If not, the datalogger just only measure the temperatures and save the data into a "data.txt" file into the sd card. If yes, it correct the measurement considering the offsets before to save the data into the sd card.
I want to reduce as much as possible what the user need to do with the data logger. Just only connect the sensors probes to each channel, introduce a sd card, power it, turn it on, and that is. One year latter, the user only need to take off the sd card, and replace for a new one, and that is. If the user does not change the sensors probe, he does not need the offset file. If one of the probes has a problem or the user want to change it, the the offset file could be used, but it is still optional, when the probe was calibrated.
Then, I think this is the easiest way to work. The user do not need to know what is inside the data logger or how it works. Just an sd card is enough. It is also fast and clean, not exposed to the environmental conditions. Originally, I designed the data logger to be used in Antarctica, where it is not easy to work outside, and neither to interact with the data logger by a notebook to upload a new firmware,... open, change the sd card, and close. Less than a minute.

Thanks!

Ok that makes sense now

So yes the parsing code you have above - Is a good place to start from

12

Thanks @MarkT,
I will study what is Malloc() and how it works.

In any case, i modified the code i had form other project that readed the input data from a bluethooth. I made the necesary changes to read the sd file. This is the new code:

#include <SPI.h>
#include <SD.h>

File myFile;
byte sensorsnumber[] = {3,3,0,0,0};                                 // Number of temperature sensors in each bus


void setup(){
  Serial.begin(9600);
  while (!Serial) {
    ;
  }
  Serial.print("Initializing SD card...");
  pinMode(10, OUTPUT);
  if (!SD.begin(10)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");
  Serial.print("Checking fie... ");
  myFile = SD.open("Offsets.txt");
  if (myFile) {
     Serial.println("file is ready");
      Serial.println("Reading the file...");
    while (myFile.available()) {                  // read from the file until there's nothing else in it:
      processIncomingByte(myFile.read());
    }
    myFile.close();
    Serial.println("  end of the file.");
  }
  else {
    Serial.println("error opening test.txt");
  }
  Serial.println("THE END");
}

void loop(){
  Serial.println("DONE");
  // nothing happens after setup
}

// Process the incoming data
void processIncomingByte (const char inChar) {
  Serial.print(inChar);
  const unsigned int MAX_INPUT = 250;  //Change to allow a longer sequence of data 250 == 30 values
  static char input_line[MAX_INPUT];
  static unsigned int input_pos = 0;
  switch (inChar) {
    case '*':
      input_line[input_pos] = 0;
      Serial.print("...processed line: ");
      Serial.println(input_line);
      process_data(input_line);
      input_pos = 0;
      break;
    default:
      if (input_pos < (MAX_INPUT - 1))
        input_line [input_pos++] = inChar;
      break;
   }
}

// Use the incoming data
void process_data (char * data) {
  int c;
  char *item[c];
  switch(data[0]){
    case 'A':
      c = sensorsnumber[0];
      break;
    case 'B':
      c = sensorsnumber[1];
      break;
    case 'C':
      c = sensorsnumber[2];
      break;
    case 'D':
      c = sensorsnumber[3];
      break;
    case 'E':
      c = sensorsnumber[4];
      break;
  }  
  char *ptr;
  int count = 0;
  ptr = strtok(data, ";");
  while (ptr) {
    item[count++] = ptr;   
    ptr = strtok(NULL, ";");
  }
  if (count < c+1){
   Serial.println("Not enough values in the array");
  }
  for (byte i = 0; i<count; i++){
    Serial.print(item[i]);
    Serial.print(" / ");
  }
}[\CODE]

This is the serial output:

Initializing SD card...initialization done.
Checking fie... file is ready
Reading the file...
A;-2.34;2.45;3.56*...processed line: A;-2.34;2.45;3.56
A / -2.34 / 2.45 / 3.56 /
B;3.45;2.0;5.55*...processed line:
B;3.45;2.0;5.55

B / 3.45 / 2.0 / 5.55 / end of the file.
THE END

The main problem i see now is that the arduino freeze, because it never goes inside the loop!! Do you see what is wrong in the code? I checked multiples times, but i can´t see what is happend.

The other problem is that i do not know why it does not recognize the end of line, and i had to use a symbol("*") to mark the end of the line, and trigger its processing.

Any idea about how to solve this issues? Thanks in advance!

13 
 int c;
char *item[c];

How many items do you think you get in your item array?

14

Thanks @J-M-L,
I really do not know. Only when the user connect the sensors probes and connect the detalogger is when the firmware knows how many. In any case, yes, I can change it to byte, because in any case it will be less than 256... I think that the limit is about 50 sensors per wire, if I remember correctly.