Data Logger new offset .txt file

Hi all, Im new with Arduino and I would really appreciate your help to finish my project, thank you in advance.
I have an Arduino Uno, a data logger shield from Adafruit and an SCD30 sensor for temperature, Co2 and humidity. The data logger code from Adafruit opens a datalog.txt file, but I need to open another .txt file with manually user defined offset values so that the data logger stores the actual sensor value +- the offset values from the other .txt file.
Example: The temperature recorded by the sensor is 22 °C and the offset value from the other text file is 2 °C, then the data logger should write 20 °C to the datalog.txt file (22 °C - 2 °C from the offset file).

Here you can also see the Data Logger skript.

const int chipSelect = 10; //10 is default by shield, but normally on Pin 4
int interval = 5;  //Log to SD Card every 5 seconds

long timer;
String timestring;
String mvalue;

RTC_PCF8523 rtc;

void setup() {
  Serial.begin(9600);
  delay(3000);
  Serial.println("Initializing SD card...");
  if (!SD.begin(chipSelect)) {
    Serial.println("SD Card error");
    return;
  }
  Serial.println("card initialized");
  if (! rtc.begin()) {
    Serial.println("No RTC found");
  } else {
    Serial.println("RTC clock found");
  }
  if (! rtc.isrunning()) {
    Serial.println("RTC is not configured");
  }
}

void loop() {
  if ((timer + interval * 1000) < millis()) {
    timer = millis();
    get_logvalue(); //Get your value
    get_time(); //Get time from RTC
    write_data(); //Write value and Time to SD
  }
}

void get_logvalue() {
  mvalue = "My Value"; //mvalue is your log parameter eg. Temperature
}

void get_time(){ //Read Time from RTC
  DateTime now = rtc.now();
  timestring = now.day();
  timestring += "-";
  timestring += now.month();
  timestring += "-";
  timestring += now.year();
  timestring += " ";
  timestring += now.hour();
  timestring += ":";
  timestring += now.minute();
  timestring += ":";
  timestring += now.second();
  Serial.println(timestring);
}

void write_data() { //Write to SD card
  String dataString = mvalue + "," + timestring;
  File dataFile = SD.open("datalog.txt", FILE_WRITE);
  if (dataFile) {
    dataFile.println(dataString);
    dataFile.close();
    Serial.println(dataString);
  }
  else {
    Serial.println("error writing datalog.txt");
  }
}

Translated with DeepL Translate: The world's most accurate translator (free version)

Why not make the offset values const variables in the sketch and add them to the values read before saving the result to the SD card ?

One argument against that is that it will require a recompile if you want to change something. EEPROM however can be a solution with some code to modify the value using e.g. serial.

Hi @UKHeliBob thank you for your time,
the values should be from user changable anytime throught the offset.txt file.

@reddevil2022, you can read the offset from a file on SD card in setup and store it in a variable and next use that variable in calculations.

How many offsets are you talking about?

one offset for each value. Temperature, CO2 and Humidity.

I've made two suggestions; take your pick :wink:

Could you please write an code example or give me some more tips where i can look and learn about it. Im pretty new with coding aswell and would appreciate your help. Thanks

Does the code that you posted compile and write data to the file ?

I have my doubts

It compiles, but it doesnt write data yet because the values from the sensor are not included yet, I have not yet soldered the sensor, but i tried them seperately and the sensor aswell as the data logger work just fine. I've been searching and thinking about the problem I have, I dont have an Idea yet how to it.

You don't actually need the sensor to test the principle as you could generate random numbers in the right range and save them

In the simplest case your sketch would open and read the file containing the offsets giving 3 variables to be used in loop() to be used as offsets to the values read by the sensor

Some things to think about

  • is the offsets file going to be on the same SD card as the log file ?
  • what should the sketch do if the offsets file does not exist ?
  • the sketch could fall back to using values previously saved to EEPROM if the file does not exist
  • should the sketch test that the offsets are within reasonable ranges given that they will be entered by a user ?
  • will the offsets be integer or float values ?
  • will the offsets file contain binary data or characters representing the offset values ?

How is the offset.txt file going to get onto the SD card?

What will be the data format of the offset.txt file?

Would it be possible to take the SD card put a offset.txt file onto the SD card. Then in setup() read the offset.txt file, putting the offsets into an array or some other thingy and then use the offsets as per the OP thingy?

This is the "full" code i have so far!

const byte pin_LED = 13;
const unsigned long timerInterval = 5000;  
unsigned long timerLast = 0;  // timer
char isoDateTime[16];
float CO2ppm = 0.0;
float TempC = 0.0;
float HumidityPct = 0.0;
#include "RTClib.h"
RTC_PCF8523 DataLoggerRTC;
#include <SPI.h>
#include <SD.h>
const int SDchipSelect = 10;  
File sdfile;
char filename[14];

#include <Wire.h>
#include "SparkFun_SCD30_Arduino_Library.h"
SCD30 airSensor;
const byte pinRDY = 8;   // Data ready pin.  High when data is ready for read-out.

void setup() {  
  Serial.begin(9600);
  while (!Serial) {
    delay(1);
  }

  //////////////////////////////////////////////////////////////////////////////
  // Sensirion SCD30 CO2 sensor
  Wire.begin();
  if (airSensor.begin() == false) {
    Serial.println("ERROR - CO2 sensor not detected.  Check wiring");
    while (1) { blinkERR(pin_LED);}
  }
  pinMode(pinRDY, INPUT);
  
  //////////////////////////////////////////////////////////////////////////////
  // Initialize the DataLogger RTC
  if (! DataLoggerRTC.begin()) {
    Serial.println("ERROR - Couldn't find DataLogger RTC");
    while (1) { blinkERR(pin_LED);}
  }
  if (! DataLoggerRTC.initialized()) {
    Serial.println("ERROR - DataLogger RTC (PCF8523) is not running!");
    while (1) { blinkERR(pin_LED);}
  } else {
    Serial.println("DataLogger RTC (PCF8523) is running");
  }
  //////////////////////////////////////////////////////////////////////////////
  // see if the SD card is present and can be initialized: 
  if (!SD.begin(SDchipSelect)) {
    Serial.println("SD card failed, or not present");
    while (1) { blinkERR(pin_LED);}
  }
  Serial.println("SD card initialized."); 
  // Create a filename reference to a file that doesn't exist 'ANALOG00.TXT'..'ANALOG99.TXT'
  //char filename[14];
  strcpy(filename, "/ANALOG00.TXT");
  for (int8_t i = 0; i < 100; i++) {
    filename[7] = '0' + i/10;
    filename[8] = '0' + i%10;
    // create if does not exist, do not open existing, write, sync after write
    if (SD.exists(filename)){
      Serial.print("File '");
      Serial.print(filename);
      Serial.println("' already exists");
    } else {
      Serial.print("New file will be '");
      Serial.print(filename);
      Serial.println("'");
      break;
    }
  }
  // Open file on SD Card for writing
  sdfile = SD.open(filename, FILE_WRITE);
  if (! sdfile) {
    Serial.print("ERROR - unable to create '");
    Serial.print(filename); Serial.println("'");
    while (1) { blinkERR(pin_LED);}
  }
  //////////////////////////////////////////////////////////////////////////////

  Serial.println("Setup complete\n");
  Serial.println("Date/Time; CO2ppm; TempC; Humidity%");
  sdfile.println("Date/Time; CO2ppm; TempC; Humidity%");
} // setup()


void loop() {
  
  if (timerLast > millis())  timerLast = millis();
  if ((millis() - timerLast) > timerInterval) {
    // toggle LED to show write activity
    digitalWrite(pin_LED, HIGH);
    DateTime now = DataLoggerRTC.now();
    //float f = fGetLgSignedRandFloat();
    CO2ppm = airSensor.getCO2();
    TempC = airSensor.getTemperature();
    HumidityPct = airSensor.getHumidity();
    sprintf(isoDateTime, "%4d%02d%02dT%02d%02d%02d%Z",
        now.year(), now.month(), now.day(), now.hour(), now.minute(), now.second());
    Serial.print(isoDateTime);
    Serial.print(";");
    Serial.print(CO2ppm);
    Serial.print(";");
    Serial.print(TempC);
    Serial.print(";");
    Serial.println(HumidityPct);

    // Write to the SD card..
    sdfile.print(isoDateTime);  
    sdfile.print(";");
    sdfile.print(CO2ppm);
    sdfile.print(";");
    sdfile.print(TempC);
    sdfile.print(";");
    sdfile.println(HumidityPct);
    // Execute a flush() to insure it is written since no sdfile.close() will be issued.
    sdfile.flush();
    digitalWrite(pin_LED, LOW);
    timerLast = millis();
  }
  
  yield();
  
} // loop()


void blinkLED(byte ledPIN){
  //  consumes 300 ms.
  for(int i = 5; i>0; i--){
    digitalWrite(ledPIN, HIGH);
    delay(30);
    digitalWrite(ledPIN, LOW);
    delay(30);
  }    
} // blinkLED()

void blinkERR(byte ledPIN){
  // S-O-S
  const int S = 150, O = 300;
  for(int i = 3; i>0; i--){
    digitalWrite(ledPIN, HIGH);
    delay(S);
    digitalWrite(ledPIN, LOW);
    delay(S);
  }    
  delay(200);
  for(int i = 3; i>0; i--){
    digitalWrite(ledPIN, HIGH);
    delay(O);
    digitalWrite(ledPIN, LOW);
    delay(O);
  }    
  delay(200);
  for(int i = 3; i>0; i--){
    digitalWrite(ledPIN, HIGH);
    delay(S);
    digitalWrite(ledPIN, LOW);
    delay(S);
  }    
  delay(200);
} // blinkERR()

Hi @UKHeliBob,
answering your questions:

  • The offset is gonna be on the same SD card as the log file.
  • if the offset file doesnt exist than the sketch should write the values we got from the sensor.
  • YES, thats also another point i gotta also set some range of values so the user wont be able to write values outside the range.
  • Offset has to be float values.
  • Offset file will contain character values.

"In the simplest case your sketch would open and read the file containing the offsets giving 3 variables to be used in loop() to be used as offsets to the values read by the sensor" - do you have any example code or any other example whatsoever that you can share with me so I can get an better idea of it. Thank you very much.

Have you looked at the examples that come with the SD library ?

Yes I did, i already know how to open and read the file but still dont know how to deduct the offset.
I was thinking to do it this way, but not sure if it works:

CO2ppm = airSensor.getCO2() - Offsetst[0];
TempC = airSensor.getTemperature() - Offsets[1];
Humidity = airSensor.getHumidity() - Offsets[2];

 sdfile.print(isoDateTime);  
    sdfile.print(";");
    sdfile.print(CO2ppm);
    sdfile.print(";");
    sdfile.print(TempC);
    sdfile.print(";");
    sdfile.println(Humidity);

In the past 2-3 days I've been doing a lot of study and research, and this is my best guess so far!

What happens when you tried?

since i cant use the sensor and Datalogger shield together cuz i have not yet soldered the sensor I took the Data Logger code from the library and gave some values for CO2, Temperatur and Humidity, it works but when I try the as above than I get the error: Invalid types 'float[int]' for array subscript.

this is the code:

#include <SPI.h>
#include <SD.h>
#include <Wire.h>
#include "RTClib.h"
#include "SparkFun_SCD30_Arduino_Library.h"
SCD30 airSensor;

const byte pin_LED = 13;
const int chipSelect = 10; //10 is default by shield, but normally on Pin 4
int interval = 5;  //Log to SD Card every 5 seconds

long timer;
String timestring;
String mvalue;
float Offsets;
float CO2ppm = 1.50;
float TempC = 1.50;
float HumidityPct = 1.50;

File sdfile; 

RTC_PCF8523 rtc;

void setup() {
  Serial.begin(9600);
  delay(3000);
  Serial.println("Initializing SD card...");
  if (!SD.begin(chipSelect)) {
    Serial.println("SD Card error");
    return;
  }
  Serial.println("card initialized");
  if (! rtc.begin()) {
    Serial.println("No RTC found");
  } else {
    Serial.println("RTC clock found");
  }
  if (! rtc.isrunning()) {
    Serial.println("RTC is not configured");
  }
}

void loop() {
  if ((timer + interval * 1000) < millis()) {
    timer = millis();
    get_logvalue(); //Get your value
    get_time(); //Get time from RTC
    write_data(); //Write value and Time to SD
  }
}

void get_logvalue() {
  //mvalue = "My Value"; //mvalue is your log parameter eg. Temperature
  Serial.print(CO2ppm)- Offsets[0];
    Serial.print(";");
    Serial.print(TempC) - Offsets[1];
    Serial.print(";");
    Serial.println(HumidityPct) - Offsets[2];
    // Write to the SD card..
    //sdfile.print(isoDateTime);  // 20200216T150644Z
    //sdfile.print(";");
    sdfile.print(CO2ppm) - Offsets[0];
    sdfile.print(";");
    sdfile.print(TempC) - Offsets[1];
    sdfile.print(";");
    sdfile.println(HumidityPct) - Offsets[2];
    Serial.println(mvalue);
}

void get_time(){ //Read Time from RTC
  DateTime now = rtc.now();
  timestring = now.day();
  timestring += "-";
  timestring += now.month();
  timestring += "-";
  timestring += now.year();
  timestring += " ";
  timestring += now.hour();
  timestring += ":";
  timestring += now.minute();
  timestring += ":";
  timestring += now.second();
  Serial.println(timestring);
}
myFile = SD.open("Offsets.txt");
  if (myFile) {
    Serial.println("checking the offsets values:");

    // read from the file until there's nothing else in it:
    while (myFile.available()) {
      Serial.write(myFile.read());
    }
    // close the file:
    myFile.close();
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening Offsets.txt");
  }
void write_data() { //Write to SD card
  String dataString = mvalue + "," + timestring;
  //String dataString = CO2ppm + ","  TempC + ","  Humidity + ","  timestring;
  File dataFile = SD.open("datalog.txt", FILE_WRITE);
  if (dataFile) {
    dataFile.println(dataString);
    dataFile.close();
    Serial.println(dataString);
  }
  else {
    Serial.println("error writing datalog.txt");
  }
}

That is useful info and should be stated early on, as a note.

Post the entire error message, please.

And the code from post#18 produces the error, correct?

Where are the offsets from the SD card put into the array?

If offsets is an array it needs to be declared as an array

float Offsets[3] = {1.1, 2.2, 3.3};

I want the values to be taken from the Offsets.txt file I have in my SD card and change them if needed without changing the code!