Go Down

Topic: SD card log file not saving correctly (Read 3882 times) previous topic - next topic

1Jesse

The log file on the SD card is not recording correctly. I verify that it is logging, blinking light, remove the card and check the file, all appears to be working fine. But when it runs for and extended period say 8 hours  and I pull the card to check the file most of it is gone. Not even the info that was there when I previously had inspected it. I think it might be that the file is not getting closed properly, here is the code it is using

Code: [Select]
//Open a file and write to it.
  File dataFile = SD.open("log.csv", FILE_WRITE);
  if ( (dataFile) && (buttonState))
  {
    dataFile.print(date);
    dataFile.print(F(","));
    dataFile.print(time);
    dataFile.print(F(","));
    dataFile.print(buttonState);
    dataFile.print(F(","));
    dataFile.print(TD);
    dataFile.print(F(","));
    //dataFile.print(val0);
    //dataFile.print(F(","));
    dataFile.print(per0);
    dataFile.print(F(","));
    //dataFile.print(val1);
    // dataFile.print(F(","));
    dataFile.println(per1);
    //dataFile.print(F(","));
    dataFile.close(); //Data isn't actually written until we close the connection!

    //Print same thing to the screen for debugging


    Serial.print(date);
    Serial.print(F("\t"));
    Serial.print(time);
    Serial.print ("\t\t");
    //Serial.print (val0);
    //Serial.print ("\t");
    Serial.print (per0);
    Serial.print ("*");
    Serial.print ("\t");
    //Serial.print (val1);
    //Serial.print ("\t");
    Serial.print (per1);
    Serial.print ("*");
    Serial.print ("\t");
    Serial.print (TD);
    Serial.print ("\t\t");
    Serial.println(buttonState);


  }
  else
  {
    Serial.println(F("Couldn't open log file"));
  }
  delay(refresh_rate);
}


Thanks for any help.

Nick_Pyner

If the card can record for that long, there can't be much wrong with the code, even though
Code: [Select]
//Data isn't actually written until we close the connection!
doesn't make much sense. Opening and closing in the loop, i.e. it is only open when it needs to be, is what I do, but I think
Code: [Select]
dataFile = SD.open("log.csv", FILE_WRITE);
should suffice and you declare it as a File up in the preamble.

It's possible that the problem is down to the way the card has been formatted. There is a sticky to that effect on this forum.

1Jesse

Thanks for your reply. I had previously read the sticky and formatted the card as recommended. I think the problem may have been my use of all the serial print lines. I have been using Jeremy Blums book and his coding, adding my own code to it. This morning I read that if the logging stops it could be that it is running out of RAM and this can be caused be excessive use of serial printing.  He suggested using the (F(code)) to utilize flash memory keeping the RAM open. I did that and also commented out the print functions. I was using them to get some calibration out of my temporary sensors until the actual thermistors arrive.

One other item is that I have not been able to read from the card. I haven't explored this very much and think it may have something to do with my use of the libraries. Although it could be an indication of a faulty shield.

Nick_Pyner

this can be caused be excessive use of serial printing. 

I think this is nonsense. You are sending about the same amount of data as me, and my stuff runs uninterrupted for months. You will have problems if you use Strings, but there is no evidence that you do.

Quote
One other item is that I have not been able to read from the card. I haven't explored this very much and think it may have something to do with my use of the libraries. Although it could be an indication of a faulty shield.
Your two problems might be connected but I would be more inclined to suspect the card than the shield. I doubt the libraries can have anything to do with it.

You might get some use out of the following. The file read should be clear, and the file write is about the same as yours.

Code: [Select]
/*
This programme reads temperatures from  three DS18B20 sensors at one
second intervals.

The temperatures are displayed on Nokia 5110 LCD 

Two of the temperatures, plus the difference between them, is transmitted
over bluetooth in a format suitable for BlueTooth Terminal/Graphics (CETIN)
in order to display three graphs in real time.

Every tenth reading is recorded on SD card in real numbers to two decimal
places. Each line is timestamped.

A new file is created at midnight using the date as the filename.

Daily files can be read via bluetooth by sending MMDD

Credit to Hacktronics, Mellis & Igoe, Garage Box, Zoomkat, Bildr.org, etc.
Kudos to lar3ry in Saskatchewan, Stanley in KL, Coding Badly in Tx.
*/

#include <OneWire.h>
#include <DallasTemperature.h>
#include <PCD8544.h>             // Nokia 5110
#include "Wire.h"                // MUST HAVE lib for LCD disp, SD card
#include <SD.h>
#include <SPI.h>                 // SD

#define DS1307_ADDRESS 0x68

char filename[] = "00000000.CSV";
File myFile;
char dumpName[] = "00000000.CSV";
File dumpFile;

static PCD8544 lcd;

// Custom symbols
static const byte DEGREES_CHAR = 1;
static const byte degrees_glyph[] = { 0x00, 0x07, 0x05, 0x07, 0x00 };
static const byte SLASH_CHAR = 2;
static const byte slash_glyph[] = {0x00,0x20,0x10,0x08};

// Green group Dee Why (red or amber LED shields)
byte InThermo[8] =  {0x28, 0x69, 0xC2, 0xB0, 0x03, 0x00, 0x00, 0x9F };
byte OutThermo[8] = {0x28, 0x7A, 0x8B, 0xC0, 0x03, 0x00, 0x00, 0x2F};
byte DrainThermo[8] = {0x28, 0x54, 0xF7, 0x2D, 0x04, 0x00, 0x00, 0x68};
 
#define ONE_WIRE_BUS 3
OneWire oneWire(ONE_WIRE_BUS);
DallasTemperature sensors(&oneWire);
 
byte  second, minute, hour, weekDay, day, month, year;
int k=0;

const int chipSelect = 4;

float tempC, InTemp, OutTemp, DrainTemp, diff;

// Define the strings for our datastream IDs
char sensorId0[] = "InThermo";
char sensorId1[] = "OutThermo";
char sensorId2[] = "DrainThermo";
char strIn[8];
char strOut[8];
char strDrain[8];
char strdiff[8];
char charBuf [13];

String readString;
String stringOne, stringTwo, stringThree, stringFour;
String stringFive, stringSix;

//_________________________________________

void setup() {
  lcd.begin(84, 48);
     // Register the custom symbols...
  lcd.createChar(DEGREES_CHAR, degrees_glyph);
  lcd.createChar(SLASH_CHAR, slash_glyph);
 
  Wire.begin();
  Serial.begin(9600);
 
  delay(300);//Wait for newly restarted system to stabilize
 
  sensors.setResolution(InThermo, 12);
  sensors.setResolution(OutThermo, 12);
  sensors.setResolution(DrainThermo, 12);
 
  lcd.setCursor (0,0);
  lcd.println("Init SD CARD");
  // make sure that the default chip select pin 53 is set to
  // output, even if you don't use it:
  pinMode(53, OUTPUT);//MEGA
 
  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect))
  {
    lcd.println("Card failed");
    // don't do anything more:
    return;
  }
  lcd.println("CARD OK");
  delay(2000);
      GetClock();
      getFileName();
      lcd.setCursor(0,3);
      lcd.println(filename);
        delay(2000);
 
    lcd.clear();
       
  //Sequence for bluetooth
  // "E256,-321,982\n"
  //so insert three variable between four strings, two are the same twice,
  //to make a fourth string
 
  stringOne = "E";
  stringTwo = ",";
  stringThree = "\n";

    running();
  }
//________________________________________________________________

void loop() {
   GetClock();
  if (hour == 0 && minute == 0 && second <2)
  {
    getFileName();
  }
 
    while (Serial.available())
  {
    delay(3); 
    char c = Serial.read();
    readString += c;
  }// end while

  if (readString.length() >0)
  { 
    getDump();   
   readString=""; 
   } // end if
 
  //get the values from the DS8B20's
  sensors.requestTemperatures();

  InTemp = sensorValue(InThermo);
  OutTemp = sensorValue(OutThermo); 
  DrainTemp = sensorValue(DrainThermo);

  diff = OutTemp - InTemp;
 
dtostrf(InTemp,4, 2, strIn);
dtostrf(OutTemp,4, 2, strOut);
dtostrf(diff,4, 2, strdiff);

stringFour = stringOne + strIn + stringTwo + strOut;
stringFour = stringFour + stringTwo + strdiff + stringThree;

  Serial.println(stringFour);

  lcd.setCursor(49,0);
  lcd.print(InTemp);
  lcd.setCursor(49,1);
  lcd.print (OutTemp);
  lcd.setCursor(49,2);
  lcd.print(DrainTemp);
  lcd.setCursor(49,3);
  lcd.print(diff); 
  lcd.print(" ");//suppress trailing fig > 9.99
  lcd.setCursor(17,5);

  lcd.print(hour);
  lcd.print(":");
  lcd.print(minute);
  lcd.print(":");
  lcd.print(second);
  lcd.print("   ");

  k=k+1; 

  if (k>9 )
  { 
           myFile = SD.open(filename, FILE_WRITE);//<<<<<<<<<<<<< OPEN
  myFile.print(hour);
  myFile.print(":");
  myFile.print(minute);
  myFile.print(":");
  myFile.print(second);
  myFile.print(",");

  myFile.print(InTemp);
  myFile.print(",");
  myFile.print(OutTemp);
  myFile.print(",");
  myFile.print(DrainTemp);
  myFile.print(",");
  myFile.print(diff);
  myFile.println();
       myFile.close();//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>CLOSE     
      k=0;
  }
  delay(850);
}  // loop ends here

//sensorValue function
float sensorValue (byte deviceAddress[])
{
  tempC = sensors.getTempC (deviceAddress);
  return tempC;
}

byte bcdToDec(byte val)  {
  // Convert binary coded decimal to normal decimal bers
  return ( (val/16*10) + (val%16) );
}

void running(){
  lcd.setCursor(0,0);
  lcd.print("In");
  lcd.setCursor(31,0);
  lcd.print("\001C ");
  lcd.setCursor(0,1);
  lcd.print("Out");
  lcd.setCursor(31,1);
  lcd.print("\001C ");
  lcd.setCursor(0,2);
  lcd.print("Drain");
  lcd.setCursor(31,2);
  lcd.print("\001C ");
  lcd.setCursor(0,3);
  lcd.print("diff");
  lcd.setCursor(31,3);
  lcd.print("\001C ");
  lcd.setCursor(15,4);
  lcd.print("Bluetooth!"); 
  }

void getFileName(){
sprintf(filename, "%02d%02d%02d.csv", year, month, day);
}

void GetClock(){
  // Reset the register pointer
  Wire.beginTransmission(DS1307_ADDRESS);
  byte zero = 0x00;
  Wire.write(zero);
  Wire.endTransmission();
  Wire.requestFrom(DS1307_ADDRESS, 7);

  second = bcdToDec(Wire.read());
  minute = bcdToDec(Wire.read());
  hour = bcdToDec(Wire.read() & 0b111111); //24 hour time
  weekDay = bcdToDec(Wire.read()); //0-6 -> sunday - Saturday
  day = bcdToDec(Wire.read());
  month = bcdToDec(Wire.read());
  year = bcdToDec(Wire.read());
}

void getDump() {
   stringSix = "2015" + readString + ".csv";
   stringSix.toCharArray(charBuf, 15); 
     Serial.println("");
     Serial.println(charBuf);
  File dumpFile = SD.open(charBuf);
  if (dumpFile)
  {
    lcd.clear();
      lcd.setCursor (0,1);
      lcd.println("DUMP FROM ");
      lcd.setCursor (0,2);
      lcd.println(charBuf);
    while (dumpFile.available())
    {
      Serial.write(dumpFile.read());
    }
    dumpFile.close();
      lcd.clear();
      running();
  } 
  else {
    Serial.println("error opening file");
       }
}

1Jesse

#4
May 31, 2015, 10:27 pm Last Edit: Jun 01, 2015, 01:54 am by 1Jesse
Thanks Nick, I have loaded the code, will go through it and do my best to learn from it. I think it is much closer to what I want to accomplish. I am an air conditioning repair man and also own a small home monitoring business. My goal here is to build a simple solution to monitor TDs and humidity along with runtimes to determine if there is a problem with the equipment.
And I will also replaced the SD card. 

Well changing the strings to the flash memory appears to have allowed the card to keep recording. Below is the code that stores the strings. I really don't like the idea of running this through the loop every time since the date only changes every 24 hours. I have several ideas in  regards to the logic, but at this time not the ability to write it down. The code you provided was great!... and way over my ability to comprehend. I am going to back it up and do more exploration into the coding side. It seems to me that the Arduino is much more code intensive than it appears upon first observation. I am considering giving the  Raspberry Pi2 a shot and see if can be of use to me in learning to program better.

void loop()
{
  //Get the current date and time info and store in strings
  DateTime datetime = RTC.now();
  year  = String(datetime.year(),  DEC);
  month = String(datetime.month(), DEC);
  day  = String(datetime.day(),  DEC);
  hour  = String(datetime.hour(),  DEC);
  minute = String(datetime.minute(), DEC);
  second = String(datetime.second(), DEC);

  //Concatenate the strings into date and time
  date = year + "/" + month + "/" + day;
  time = hour + ":" + minute + ":" + second;

Nick_Pyner

You might have a look at the getclock and file.write in mine. Note that no library is used for time, what you see is all the there is to be seen.

1Jesse

I am still having a problem writing to an SD card and would grateful for some assistance.
 The Arduino has to be attached to a computer and it appears it also needs to be writing to the serial port. The Arduino will write to the SD card with the following code....
 
Code: [Select]
#include <Wire.h>
#include <DallasTemperature.h>
#include <OneWire.h>
#include <SD.h>
#include <RTClib.h>
#include <SPI.h>

const int CS_PIN      = 10;
const int SD_POW_PIN  = 8;   ///FOR RTC
const int RTC_POW_PIN = A3;
const int RTC_GND_PIN = A2;
const int contacts = 2;
float TD, InTemp, OutTemp;
//Default rate of 5 seconds
int refresh_rate = 5000;

//Define an RTC object
RTC_DS1307 RTC;
OneWire oneWire(7);
DallasTemperature sensors(&oneWire);
//Initialize strings
String year, month, day, hour, minute, second, time, date;
float currentTemp0;
float currentTemp1;
void setup()
{
  Serial.begin(9600);
  Serial.println(F("Initializing Card"));
  sensors.begin();
  //CS pin, and pwr/gnd pins are outputs
  pinMode(CS_PIN,   OUTPUT);
  pinMode(SD_POW_PIN, OUTPUT);
  pinMode(RTC_POW_PIN, OUTPUT);
  pinMode(RTC_GND_PIN, OUTPUT);

  //Setup power and ground pins for both modules
  digitalWrite(SD_POW_PIN, HIGH);
  digitalWrite(RTC_POW_PIN, HIGH);
  digitalWrite(RTC_GND_PIN, LOW);

  pinMode(contacts, INPUT);

  //Initiate the I2C bus and the RTC library
  Wire.begin();
  RTC.begin();

    if (! RTC.isrunning())
  {
    Serial.println(F("RTC is NOT running!"));
    RTC.adjust(DateTime(__DATE__, __TIME__));
  }

  //Initialize SD card
  if (!SD.begin(CS_PIN))
  {
    Serial.println(F("Card Failure"));
    return;
  }
  Serial.println(F("Card Ready"));

  int refresh_rate = 5000;
  //Write column headers
  File dataFile = SD.open("log.csv", FILE_WRITE);
  if (dataFile)
  {
    dataFile.println(F("\nNew Log Started!"));
    dataFile.println(F("Date,Time,Phrase"));
    dataFile.close(); //Data isn't actually written until we close the connection!
    Serial.println(F("\nNew Log Started!"));  //Print same thing to the screen for debugging
    Serial.println(F("Date,Time,Phrase"));
  }
  else
  {
    Serial.println(F("Couldn't open log file void setup"));
  }
}

void loop()
{
  //Get the current date and time info and store in strings
  DateTime datetime = RTC.now();
  year  = String(datetime.year(),  DEC);
  month = String(datetime.month(), DEC);
  day  = String(datetime.day(),  DEC);
  hour  = String(datetime.hour(),  DEC);
  minute = String(datetime.minute(), DEC);
  second = String(datetime.second(), DEC);

  date = year + "/" + month + "/" + day;    //Concatenate the strings into date and time
  time = hour + ":" + minute + ":" + second;

  int buttonState = digitalRead(contacts);
  sensors.requestTemperatures();
  currentTemp0 = sensors.getTempCByIndex(0);
  currentTemp1 = sensors.getTempCByIndex(1);
  currentTemp0 = currentTemp0 * 1.8 + 32;
  currentTemp1 = currentTemp1 * 1.8 + 32;
  TD  = currentTemp1 - currentTemp0;

  File dataFile = SD.open("log.csv", FILE_WRITE);  //Open a file and write to it.
  if ((dataFile) && (buttonState))
  {
    dataFile.print(date);
    dataFile.print(",");
    dataFile.print(time);
    dataFile.print(",");
    dataFile.print(currentTemp0);
    dataFile.print(",");
    dataFile.print(currentTemp1);
    dataFile.print(",");
    dataFile.print(TD);
    dataFile.println(",");
    dataFile.close();                   
   
    Serial.print(date);           THIS IS THE CODE
    Serial.print(F("\t"));        I HAVE TO COMMENT OUT.
    Serial.print(time);
    Serial.print(F("\t"));
    Serial.print("Return - ");
    Serial.print(currentTemp1);
    Serial.print("\t\tSupply - ");
    Serial.print(currentTemp0);
    Serial.print("\tTD - ");
    Serial.println(TD);
  }
  else
  {
    Serial.println(F("Couldn't open log file void loop"));
  }
  delay(refresh_rate);
}


... if the lines that serial.print  the date, time, temp and TD are commented out it will not write to the SD card. If the lines are left in it will write the data to the card, but it still has to be attached the computer.
 I have replaced the SD data shield and the RTC. The sensors are giving me accurate readings.  Thanks for any help.

Nick_Pyner

#7
Jun 09, 2015, 04:59 am Last Edit: Jun 09, 2015, 06:26 am by Nick_Pyner
I'm surprised that you get any result at all, but not surprised at what you don't get. I believe the first thing you need to do is explain why you have the clock and the SD card wired up in manner that nobody else has, or likely needs to.

If you have read the code I posted at all, you might be a bit surprised by it but I assure you it is pretty normal.

Maybe you have a different setup but, even if it is just an SD module, I don't see why you should vary the standard code.

Code: [Select]
const int chipSelect = 4;   

void setup()

  pinMode(10, OUTPUT);     // change this to 53 on a mega


and I don't think the clock merits discussion at the moment.

cattledog

Power consumption by an SD card module is quite high, and Vcc for the module should not be sourced from an output pin.

I have connected ground and power on an RTC from A2/A3 when I had four male pins plugged into A2/A3 for power and A4/A5 for SDA/SCL. That should be OK.

I don't know about the interaction with the Serial.print statements, but first, correct the power supply for the SD module.

1Jesse

Thanks for the replies gentlemen. The coding for the  SD card reader and RTC were copied from a book I purchased to help me learn and understand the Arduino. So Iassumed it was good and standard coding. I have a full day at my regular job but will repost after changing the power supply and the other code.

Go Up