Unable to store more than 16 pieces of data into SD card

[tlyao]

Hi, I'm a rookie in Arduino. Recently, I'm assembling a CO2 sensor (SCD30), RTC (DS3231) and a microSD card adaptor with an Arduino Nano board. It seems a simple configuration, but somehow only 16 pieces of data can be written into SD card. All the board, wires and devices were checked or even replaced by a new one. This annoying problem is still unsolved. Anybody can give me some ideas what's going on? The code is as below,

#include <stdio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <DS3231.h>
#include <SparkFun_SCD30_Arduino_Library.h> //Click here to get the library: http://librarymanager/All#SparkFun_SCD30

SCD30 airSensor;

File myFile;

DS3231 myRTC(SDA, SCL);

const int chipSelect = 10;
unsigned int DAY;
unsigned int MONTH;
unsigned int YEAR;
unsigned int HOUR;
unsigned int MINUTE;
unsigned int SECOND;
unsigned int CO2DATA;
float TEMPDATA;
float HUMIDDATA;
boolean AVAILABLE;


void setup() {
  Serial.begin(115200); //baud rate: 115200
  Wire.begin(); //wire setup for CO2 sensor
  myRTC.begin();


  // The following lines can be uncommented to set the date and time
  //myRTC.setDOW(TUESDAY);     // Set Day-of-Week to SUNDAY
  myRTC.setTime(12, 0, 0);     // Set the time to 12:00:00 (24hr format)
  myRTC.setDate(31, 8, 2021);   // Set the date to August 31st, 2021
  


  //CO2 sensor initialize
  Serial.println("Initializing SCD30 ...");
  if (airSensor.begin() == false) {
    while (1) {
      Serial.println(F("Air sensor not detected. Please check wiring and reset ARDUINO to try again."));
      delay(5000);
    }
  }
  Serial.println(F("SCD30 successfully initialized!"));


  //SD card initialize
  Serial.println(F("Initializing SD card ..."));
  if (!SD.begin(10)) {
    while (1) {
      Serial.println(F("SD card initialization failed. Please check wiring and reset ARDUINO to try again."));
      delay(5000);
    }
  }
  Serial.println(F("SD card successfully initialized!"));
}



void loop() {
  myRTC.getTime(); //This allows for the update of variables for time or accessing the individual elements.

  if (airSensor.dataAvailable()) {
    CO2DATA   = airSensor.getCO2();
    TEMPDATA  = airSensor.getTemperature();
    HUMIDDATA = airSensor.getHumidity();
    AVAILABLE = 1;
  }
  else {
    AVAILABLE = 0;
  }
  

  //print time to computer
  Serial.print("Current Date / Time: ");
  Serial.print(myRTC.getDateStr());
  Serial.print(" ");
  Serial.print(myRTC.getTimeStr());


  //print C02 measurement to computer
  if (AVAILABLE) {
    Serial.print(" co2(ppm) ");
    Serial.print(CO2DATA);

    Serial.print(" temp(C) ");
    Serial.print(TEMPDATA, 1);

    Serial.print(" humidity(%) ");
    Serial.print(HUMIDDATA, 1);

    Serial.println();
  }
  else {
    Serial.println("Waiting for new data");
  }


  //print date and C02 measurement to SD card
  myFile = SD.open("test.txt", FILE_WRITE);
  if (myFile) {
    Serial.println(F("Writing to test.txt..."));

    myFile.print("Current Date / Time: ");
    myFile.print(myRTC.getDateStr());
    myFile.print(" ");
    myFile.print(myRTC.getTimeStr());

    if (AVAILABLE) {
      myFile.print(" co2(ppm) ");
      myFile.print(CO2DATA);

      myFile.print(" temp(C) ");
      myFile.print(TEMPDATA, 1);

      myFile.print(" humidity(%) ");
      myFile.print(HUMIDDATA, 1);

      myFile.println();
    }
    else {
      myFile.println("CO2 data not available.");
    }
    
    //close the file
    myFile.flush();
    Serial.println(F("DATE and DATA had been saved into test.txt!"));
  } 
  else {
    //if the file didn't open, print an error
    Serial.println(F("error opening test.txt"));
  }

  delay(5000);
}

[UKHeliBob]

You never close the file in the sketch. Could that cause a problem ?

Have you considered using FILE_APPEND rather than FILE_WRITE ?

[tlyao]

Yes, it goes well after changing flush() to close(). Thank you so much for the help.
But it’s still confusing why the number 16? Before the revision, the 17th storage always went wrong. Is it some sort of traffic jam of data transmission?

[sterretje]

@tlyao

But it’s still confusing why the number 16?

Quite possible that that is the number of file handles that is either supported or makes your application run out of memory.

[tlyao]

Thank you for the reply.
But the data file is only a few KB and the SD card is 16G.

[sterretje]

I said "file handles"; that is not the size of the files or the number of files on the card.

When you open a file with a statement like myFile = SD.open("test.txt", FILE_WRITE);, myFile is a file handle. If you don't close it, it will stay there. Next time you open a file, another file handle is created. File handles consume memory, so that can be one reason. There might also be a limit in the library although in that case I would expect the below to actually tell you.

  else {
    //if the file didn't open, print an error
    Serial.println(F("error opening test.txt"));
  }

[tlyao]

It does show “error opening test.txt” on the monitor when the data cannot be stored into SD card, so do you mean that it is more possible that some limit in the library caused this problem?
By the way, at first, I thought it might be related to hexadecimal error. Just a wild guess.

[UKHeliBob]

It is not a limit in the library beyond the fact that you can only have one file open at a time as noted in the examples with the SD library

  // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  myFile = SD.open("test.txt", FILE_WRITE);

[cattledog]

As of version 1.0, the library supports opening multiple files.
https://www.arduino.cc/en/reference/SD

As @sterretje notes, opening 16 files can likely create issues with file handles and possible memory buffer reserves.

[tlyao]

I see. Thank you all for the help. It really gives me a clearer picture of the data storage procedures.