Strangeness opening files on SD card

Ok, I’ve run into what appears to be a simple roadblock once found, and encountered something really strange! I’m building a volts/amps/temps datalogger that will log six different values. The hardware consists of an Arduino Mega Pro, HC-06 Bluetooth, MicroSD card, an RTC clock module, a ZMPT101B AC voltage sensor, an ACS712 current sensor, and an SCT-013-030 current sensor. I’ve set it up so that it will store six values in a file on the SD card that indicate which of the values to actually log, along with calibration values for some of the sensors.

A function named “loadConfig()” was added to load the float values from the “/CONFIG.TXT” file on the SD card into variables, and it’s called early on in setup(). When it runs, it’s unable to open the “/CONFIG.TXT” file. There’s another routine that opens the same config file when I send it a command via Bluetooth to send the contents of the config file to my phone, and that one has worked without fail. So, as usual, I copied the bare basics into a new sketch, compiled it, and it runs perfectly. So, the first question is, why would the app open the config file when sending the data via Bluetooth but not when it attempts to load the files in the loadConfig routine? The second question is, why would the exact same code run in the standalone sketch and not as part of the main application?

When the main program is compiled, it shows using 19% of the program memory and 53% of the dynamic memory, so I’m feeling that memory isn’t an issue:

Sketch uses 50138 bytes (19%) of program storage space. Maximum is 253952 bytes.
Global variables use 4411 bytes (53%) of dynamic memory, leaving 3781 bytes for local variables. Maximum is 8192 bytes.

Here’s the small sketch containing the code that works. Everything was copied and pasted from the main application:

#define sdCSPin 53
#include <SD.h>
File configFile;
const int numFields=34;
float numValue[numFields];
float SENSITIVITY;
double ampsCalibrationValue;
float R1, R2;

void setup() {
  Serial.begin(115200);
  SD.begin(sdCSPin);
  loadConfig();
}

void loop() {
  // put your main code here, to run repeatedly:

}

void loadConfig() {
  configFile = SD.open("/CONFIG.TXT");
  Serial.println(configFile);
  if (configFile) {
    while (configFile.available()) {
      numValue[27]=configFile.parseFloat();
      numValue[28]=configFile.parseFloat();
      numValue[29]=configFile.parseFloat();
      numValue[30]=configFile.parseFloat();
      numValue[31]=configFile.parseFloat();
      numValue[32]=configFile.parseFloat();
      SENSITIVITY=configFile.parseFloat();
      ampsCalibrationValue=configFile.parseFloat();
      R1=configFile.parseFloat();
      R2=configFile.parseFloat();

      Serial.println(numValue[27]);
      Serial.println(numValue[28]);
      Serial.println(numValue[29]);
      Serial.println(numValue[30]);
      Serial.println(numValue[31]);
      Serial.println(numValue[32]);
      Serial.println(SENSITIVITY);
      Serial.println(ampsCalibrationValue);
      Serial.println(R1);
      Serial.println(R2);
    }
    configFile.close();
  } else {
    Serial.println("Error opening /CONFIG.TXT");
  }
  while (1);
}

When it runs, it displays the status of “configFile” along with the ten values in the config file:

image

Now the strange part! Approximately 9-10 seconds after the ten values are displayed, ten more values of “0.00” appear right below the first ten values. The file is closed after the contents have been read, so I have no idea what’s generating the second set of values. I occasionally forget to dimension arrays large enough and that definitely generates some interesting results, but I’ve never come across this situation before.

Anyone care to take a swing at this? Thanks!

Update: I added a line of code to verify that the file was closed before the second set of values was displayed, and it wasn’t!

#define sdCSPin 53
#include <SD.h>
File configFile;
const int numFields=34;
float numValue[numFields];
float SENSITIVITY;
double ampsCalibrationValue;
float R1, R2;

void setup() {
  Serial.begin(115200);
  SD.begin(sdCSPin);
  loadConfig();
}

void loop() {
  // put your main code here, to run repeatedly:

}

void loadConfig() {
  configFile = SD.open("/CONFIG.TXT");
  Serial.println(configFile);
  if (configFile) {
    while (configFile.available()) {
      numValue[27]=configFile.parseFloat();
      numValue[28]=configFile.parseFloat();
      numValue[29]=configFile.parseFloat();
      numValue[30]=configFile.parseFloat();
      numValue[31]=configFile.parseFloat();
      numValue[32]=configFile.parseFloat();
      SENSITIVITY=configFile.parseFloat();
      ampsCalibrationValue=configFile.parseFloat();
      R1=configFile.parseFloat();
      R2=configFile.parseFloat();

      Serial.println(numValue[27]);
      Serial.println(numValue[28]);
      Serial.println(numValue[29]);
      Serial.println(numValue[30]);
      Serial.println(numValue[31]);
      Serial.println(numValue[32]);
      Serial.println(SENSITIVITY);
      Serial.println(ampsCalibrationValue);
      Serial.println(R1);
      Serial.println(R2);
    }
    configFile.close();
    Serial.println("file closed");
  } else {
    Serial.println("Error opening /CONFIG.TXT");
  }
  while (1);
}

Still doesn’t explain with the second set of values came from. I have very limited experience with using SD cards with the Arduino, so I’m totally open to any suggestions on how to make improvements!

The only thing that strikes me is an array of 34 or so float values is huge and you maybe getting memory issues once it run ?
Try with a much smaller array and see if it’s ok ?

Do you need floats ? Do you need an array ? What will you do with these numbers ?

You are repeating setup(). That is often indicative a a power problem causing the reset. I think the second set is 0's because it is opening the second time at the end of the file.
Or perhaps it is a memory issue causing the reset.

EDIT:
On second thought, you are not repeating setup() but reading 0 information in the while() loop. You may need to manage end of file better.

All of the variables are created at the global level. Based on what’s shown when it’s compiled, it’s barely using half of what’s available.

Seems strange that the entire application works perfectly, with the exception of opening the config file to load the ten values, especially when the exact same “configFile = SD.open("/CONFIG.TXT");” is executed when it sends the contents of the config file via Bluetooth.

I’m experimenting with using arrays to control most everything in the application as far as the OLED display is concerned. I’m using a 2.42” OLED that gives me 6 lines of 21 characters, which works perfectly for this application. The arrays contain x and y coordinates, field types, field formats, etc. It allows using “for” commands to display labels and field values on the display. I specify the starting and end field numbers, and then simple routines use the array values to format and display everything. It makes it extremely easy to add new screens, as well as re-using the same set of field information in different screens rather than starting from scratch. It also saves scrounging through the code to change things when I move things around on the screen. I just update the arrays with the new values. It takes more memory for the arrays, but it has greatly shrunk the code to accomplish everything.

I’m not following because loadConfig() is called once from setup().

Isn’t setup() executed once upon bootup, and then it loops forever in loop()? I’m only seeing the output of “Serial.println(configFile);” once in what’s displayed on the serial monitor. It’s indicated by the red arrow in the screen shot below.

I just now added a “while (1);” to the end of setup() to prevent for progressing any further, and it’s still displaying the second set of ten values of “0.00”.

#define sdCSPin 53
#include <SD.h>
File configFile;
const int numFields=34;
float numValue[numFields];
float SENSITIVITY;
double ampsCalibrationValue;
float R1, R2;

void setup() {
  Serial.begin(115200);
  SD.begin(sdCSPin);
  loadConfig();
  while(1);
}

void loop() {
  // put your main code here, to run repeatedly:

}

void loadConfig() {
  configFile = SD.open("/CONFIG.TXT");
  Serial.println(configFile);
  if (configFile) {
    while (configFile.available()) {
      numValue[27]=configFile.parseFloat();
      numValue[28]=configFile.parseFloat();
      numValue[29]=configFile.parseFloat();
      numValue[30]=configFile.parseFloat();
      numValue[31]=configFile.parseFloat();
      numValue[32]=configFile.parseFloat();
      SENSITIVITY=configFile.parseFloat();
      ampsCalibrationValue=configFile.parseFloat();
      R1=configFile.parseFloat();
      R2=configFile.parseFloat();

      Serial.println(numValue[27]);
      Serial.println(numValue[28]);
      Serial.println(numValue[29]);
      Serial.println(numValue[30]);
      Serial.println(numValue[31]);
      Serial.println(numValue[32]);
      Serial.println(SENSITIVITY);
      Serial.println(ampsCalibrationValue);
      Serial.println(R1);
      Serial.println(R2);
    }
    configFile.close();
    Serial.println("file closed");
  } else {
    Serial.println("Error opening /CONFIG.TXT");
  }
  while (1);
}

I’m not aware of another way of closing a file other than issuing the close command when it reaches the end of the file. Can you provide sample code of the method you’re referring to? Thanks!

I think I found the issue! Back shortly after some testing… ;-)

I guess I just needed to stare at it a little longer!

I’m used to “looping” through a file to read the contents. In this case, it wasn’t needed. After opening the file, it just needed to a “parseFloat” command to transfer the values into the variables. After making the adjustments, it now behaves as it should, which is to transfer the variables and close the file.

However! I copied the loadConfig() routine back into the application, and it still is not opening the file. I am now beginning to see issues that could be caused by a memory overrun. The display is displaying some strange things, so now I’m looking into the arrays and what might be going on.

#define sdCSPin 53
#include <SD.h>
File configFile;
const int numFields=34;
float numValue[numFields];
float SENSITIVITY;
double ampsCalibrationValue;
float R1, R2;

void setup() {
  Serial.begin(115200);
  SD.begin(sdCSPin);
  loadConfig();
}

void loop() {
  // put your main code here, to run repeatedly:

}

void loadConfig() {
  configFile = SD.open("/CONFIG.TXT");
  Serial.println(configFile);
  if (configFile) {
    numValue[27]=configFile.parseFloat();
    numValue[28]=configFile.parseFloat();
    numValue[29]=configFile.parseFloat();
    numValue[30]=configFile.parseFloat();
    numValue[31]=configFile.parseFloat();
    numValue[32]=configFile.parseFloat();
    SENSITIVITY=configFile.parseFloat();
    ampsCalibrationValue=configFile.parseFloat();
    R1=configFile.parseFloat();
    R2=configFile.parseFloat();
    configFile.close();
    Serial.println("file closed");
    Serial.println(numValue[27]);
    Serial.println(numValue[28]);
    Serial.println(numValue[29]);
    Serial.println(numValue[30]);
    Serial.println(numValue[31]);
    Serial.println(numValue[32]);
    Serial.println(SENSITIVITY);
    Serial.println(ampsCalibrationValue);
    Serial.println(R1);
    Serial.println(R2);
  } else {
    Serial.println("Error opening /CONFIG.TXT");
  }
}

That means you had at least a trailing byte (a CR / LF may be on the last line) after the last parseFloat that made the available() call still true and so you were looping abs then you were attempting to read garbage. The delay you were seing is likely the timeout of the various parseFloat.


Unless you share that code, we can’t help…

The problem appears to be the memory getting overrun by my heavy use of arrays. I was using them for everything, including displaying all of the labels for the fields. I’m cutting back to using them only for storing the field values.

I’m using a rotary encoder for screen, menu, and field navigation, and manipulating field values. The arrays streamline the process and greatly reduce the amount of code. The arrays also make it easy to add additional items. It’ll just take a couple of hours to get back on track.

Thanks for the input everybody! Much appreciated.

Make sure you use the F() macro for all your string literals - you’ll save some RAM

I was actually reading about that this morning! Thanks…