Using a SD/EEPROM file like a data base

My project is a Doll House, specifically the lighting of led in a number of rooms.
I have the sketch in Wokwi [Doll House Light - Wokwi Arduino and ESP32 Simulator]
( I know the SD is not correctly connected yet).

The file should contain the state of each led etc., like a config file which will be used when starting up.

My problem/question is what is a good way to store the data should - use enum or Structure or even a txt file.


const uint8_t  brightnessMin = 0;                 //  minimum brightness
const uint8_t  brightnessMax = 255;               //  maximum brightness
const uint8_t  brightnessInc = 10;                //  increase brightness by
uint8_t  brightness = brightnessMax;              //  container for current brightness
uint8_t  currentButton = -1;                      //  container for current button pressed
uint16_t received = 0;                            //  received container

// TIME
const unsigned long SamplePeriod = 1000;          // sampling period in milliseconds
const uint16_t long delay_1 = 1000;               // delay used for testing

How many records do you intend to store on the SD?
From the variable definitions, you apparently intend on updating some data elements and storing them: apparently by overwriting the old data.

The nature of SD flash will make this rewrite operation slower than sequential writes as you are effectively doing random record updates.

A far better approach would be external flash IC storage or even better FRAM.

Alternately, just keep the power on the Arduino and maintain all your states in SRAM.

Thanks for your reply, I would cap the number of records at 20.
Since this is on a nano I didn't think it would have enough memory.
Also I am not familiar Using SRAM.

You are already using SRAM. That is where all program variables are stored.

The suggestion was to keep the Arduino on, and your program running at all times.

You can store quite a bit of data in EEPROM and it will be maintained during power off. One byte of EEPROM would store the on/off states of 8 LEDs, for example.

Arduino EEPROM reference guide

here's a quick example of using a structure to save and load the config..

Save Config SD

SD really seems like overkill given the small amount of data.

Thi swould not be do able.
Thanks for the eeprom info I have looked into this and thought that with my MCU being a nano the would not be enough memory.

Thankyou for the guide, I will look into this ...

Further to my project I am trying out using EEPROM and am having some difficulty perhaps someone could assist.

https://wokwi.com/projects/357549705431599105

Perhaps if you explained what you mean by "database" and how you access the individual entries that might help.

i was unsure as to how to ; use enum or structure to read and write the information using either a SD card or EEPROM.
From what has been posted here I will use structure.(as a database record).
It is suggested here that using a SD card for 20 records should be reconsidered.
So I would like to used EEPROM.
Hope this answers your question.

The records are accessed by IR transmittter.

No. With a "database" you supply an interface program with some key data that may be part of the database indexing structure. The interface program returns that matching data or an indicator that the data does not exist. If more than one database entry matches the key data, the subsequent matches will be returned one at a time.

OK so the use of the 'word' database is incorrect.- It was used by me for the want of a better word.

An SD card file is a continuous set of records, one after another. There are many creating record for the file and updating them. You could even create a secondary file of indexing data from the records and include the record number in the first file. Then you could sort the second file to some order and use those sorted records to give you the "record number" in the first file and then you could randomly read that record.
Lots of ways to skin the cat. I just followed our cat through the snow to neighboring property 1/4 mile away. Never did find him. Have to wait till he gets hungry!

I hope your cat comes home as I am sure he/she will.
Your response is much appreciated and informative, however it further confuses me as to which is the best way to proceed.

Yeah, he came back home. His first home was with a young lady in a small trailer on a big cattle ranch about 25 miles East of here. There were lots of barns and other buildings there. We don't have that many and I think he was looking for familiar territory.
The minimum memory buffers needed for any SD file is 512 bytes for the directory and another 512 bytes reserved for your data, even if you use only one byte.
Suggest you make an array of structures for your project. You mentioned 20, so 20 entries of your control structure. All stored in EPROM and returned from EPROM when your program is started. That will take up much less Arduino memory than SD card.

eeprom are read and written a byte at a time..
try this..


#include <EEPROM.h>

struct {
  uint16_t code;                                  // associated BUTTON key hex
  byte led[];                                     // which led
  byte state;                                     // state on or off
  uint8_t brightness;                             // brihtness
} Config;

uint8_t buff[20];
char Data;

// buttons
uint8_t previousBut1 = 0;
uint8_t But1  = A0;
uint8_t previousBut2 = 0;
uint8_t But2  = A1;
uint8_t previousBut3 = 0;
uint8_t But3  = A2;


void setup() {

  Serial.begin(115200);
  Serial.println("Waiting  ");

  pinMode(But1, INPUT_PULLUP);
  pinMode(But2, INPUT_PULLUP);
  pinMode(But3, INPUT_PULLUP);

  previousBut1 =  digitalRead(But1);
  previousBut2 =  digitalRead(But2);
  previousBut3 =  digitalRead(But3);
}

void loop() {

  byte currentState = digitalRead(But1);
  if (currentState != previousBut1) {
    previousBut1 = currentState;
    if (!currentState) eClear();
  }

  currentState = digitalRead(But2);
  if (currentState != previousBut2) {
    previousBut2 = currentState;
    if (!currentState) eWrite();
  }

  currentState = digitalRead(But3);
  if (currentState != previousBut3) {
    previousBut3 = currentState;
    if (!currentState) eRead();
  }
}//--------------------------------------------------------------//


void eClear() {
  for (int x = 0; x < 20; x ++) {
    EEPROM.write(x, " ");
  }

  Serial.println("EEPROM memory is Erased.");
  delay(500);
}


void eWrite() {

Serial.println("Writing");
  for (int x = 0; x < 20; x++) {

    Config.code = 0x20+x;
    Config.led[x] = x;
    Config.state = 1;
    Config.brightness = 255;

    memcpy(&buff, &Config, sizeof(Config));
//write one byte at a time..
for (int i = 0; i<sizeof(Config);i++){
    EEPROM.write(i+(x*20), buff[i]);
    Serial.print(x );
    Serial.print(" : ");Serial.print(i);
    Serial.print(" - ");
    Serial.println(buff[i],DEC);
}
    delay(500);
  }

}

void eRead() {

  String readData;

  for (int x = 0; x < 20; x++) {
    ZeroBuff();//out with the old..

    for (int i = 0; i < sizeof(Config); i++){
      buff[i]= EEPROM.read(i+(x*20));
    }
    memcpy(&Config, &buff, sizeof(Config));
    Serial.print("Record:");
    Serial.println(x);   
    Serial.print("Code:"); 
    Serial.println(Config.code);

  }
}

void ZeroBuff(){
  for (int i = 0; i< sizeof(buff); i++)
  buff[i] = 0;
}


Glad he came home, I am struggling with a simple, clear,write, read EEPROM sample.
I have some difficulty understanding most of the examples that I have googled. I learn better by doing so if someone could look at my last sample wokwi and help me I would appreciate it.

I am struggling with a simple, clear,write, read EEPROM sample.

Start with the explanation and simple examples here: https://docs.arduino.cc/learn/built-in-libraries/eeprom

They could hardly be simpler and clearer.

I did look at your eeprom example, corrections i posted should have worked..
tinkering with it, eeproms are a bit slower than SD..

here's another example of using SD.
this one stores a max of 5 records in one file..
then it overwrites oldest to newest..

the over write bit took me a while, tricky..

#include <SD.h>

#define CS_PIN 10
//How Many Records to store in DB
#define MAX_RECS  5

#define BTN_SAMPLE 3
//random write mode
#define FILE_RANDOM_WRITE (O_WRITE | O_READ | O_CREAT)

//The Data to store
struct __attribute__((__packed__)) {
  int16_t sequence = 0;//used in function KEEP
  int16_t temp = 0;
  int16_t humidity = 0;
} SampleRec;


File root;
File myFile;

uint8_t buff[100];

uint16_t NextSequence = 1;
uint16_t CurrRec = 0;

unsigned long lastSample = 0;
int intervalSample = 20;

char command[80];
int CharCount = 0;
bool HaveCommand = false;

void setup() {
  Serial.begin(115200);

  Serial.print("Initializing SD card... ");

  if (!SD.begin(CS_PIN)) {
    Serial.println("Card initialization failed!");
    while (true);
  }

  Serial.println("initialization done.");

  Serial.println("Files in the card:");
  root = SD.open("/");
  printDirectory(root, 0);
  Serial.println("");



  NextSequence = LoadSequence();
  SampleRec.sequence = NextSequence;
  SampleRec.temp = 80;
  SampleRec.humidity = 60;
  SaveSample();
  SampleRec.sequence = NextSequence;
  SaveSample();

  Serial.println("Files in the card:");

  root = SD.open("/");
  printDirectory(root, 0);
  Serial.println("");
  pinMode(BTN_SAMPLE, INPUT_PULLUP);


}



bool Saving = false;
void loop() {
  // nothing happens after setup finishes.
  if (millis() - lastSample >= intervalSample) {
    lastSample = millis();
    if (!digitalRead(BTN_SAMPLE) && !Saving)
    {
      Saving = true;
      SampleRec.sequence = NextSequence;
      SampleRec.temp = 81;
      SampleRec.humidity = 66;
      SaveSample();
      Serial.println("Files in the card:");
      root = SD.open("/");
      printDirectory(root, 0);
      Serial.println("");

    } else if (digitalRead(BTN_SAMPLE)) Saving = false;

  }


  while (Serial.available())
  {
    char ch = Serial.read();
    if (ch != 10) {
      if (CharCount < sizeof(command)) command[CharCount] = ch;
      CharCount++;
    } else HaveCommand = true;
  }
  if (HaveCommand) {
    switch (command[0]) {
      case 'L': Serial.println("Files in the card:");
        root = SD.open("/");
        printDirectory(root, 0);
        Serial.println("");
        break;
      case 'C': Serial.print("Records: "); Serial.println(CountSamples()); break;
      case '1': LoadSample(0); Serial.println("Sample 1"); Serial.print("Seq:");
        Serial.println(SampleRec.sequence); break;
      case '2': LoadSample(1); Serial.println("Sample 2"); Serial.print("Seq:");
        Serial.println(SampleRec.sequence); break;
      case '3': LoadSample(2); Serial.println("Sample 3"); Serial.print("Seq:");
        Serial.println(SampleRec.sequence); break;
      case '4': LoadSample(3); Serial.println("Sample 4"); Serial.print("Seq:");
        Serial.println(SampleRec.sequence); break;
      case '5': LoadSample(4); Serial.println("Sample 5"); Serial.print("Seq:");
        Serial.println(SampleRec.sequence); break;
    }
    HaveCommand = false;
    CharCount = 0;
  }
}


void printDirectory(File dir, int numTabs) {
  while (true) {

    File entry =  dir.openNextFile();
    if (! entry) {
      // no more files
      break;
    }
    for (uint8_t i = 0; i < numTabs; i++) {
      Serial.print('\t');
    }
    Serial.print(entry.name());
    if (entry.isDirectory()) {
      Serial.println("/");
      printDirectory(entry, numTabs + 1);
    } else {
      // files have sizes, directories do not
      Serial.print("\t\t");
      Serial.println(entry.size(), DEC);
    }
    entry.close();
  }
}


//how many records..
int CountSamples() {
  int count = 0;
  myFile = SD.open("sample.db", FILE_READ);
  if (myFile)
  {
    if (myFile.size() > 0) {
      count = myFile.size() / sizeof(SampleRec);
    }
    myFile.close();
  }
  return count;
}

//loads the next sequence
//for start up..
int LoadSequence() {
  int seq = 0;
  int count = CountSamples();
  if (count > 0) {
    for (int i = 0; i < count ; i++) {
      if (LoadSample(i)) {
        if (SampleRec.sequence > seq) {
          seq = SampleRec.sequence;
          CurrRec = i + 1;
        }
      } 
    }
  }
  return seq + 1;
}


bool LoadSample(int recNo) {
  int count = 0;
  bool GoodRead = false;
  myFile = SD.open("sample.db", FILE_READ);
  if (myFile)
  {
    if (myFile.size() > 0) {
      if ( recNo <= myFile.size() / sizeof(SampleRec)) {
        myFile.seek(recNo * sizeof(SampleRec));
        int bytesRead = myFile.read(buff, sizeof(SampleRec));
        if (bytesRead == sizeof(SampleRec)) {
          memcpy(&SampleRec, &buff, sizeof(SampleRec));
          GoodRead = true;
        } 
      } 
    }
    myFile.close();
  } else Serial.println("File failed to open");
  return GoodRead;
}

bool SaveSample() {
  myFile = SD.open("sample.db", FILE_RANDOM_WRITE);
  if (myFile)
  {
    if (myFile.size() < sizeof(SampleRec)* MAX_RECS)
    {
      int NewPos = myFile.seek(myFile.size());

    } else
    { if (CurrRec == MAX_RECS) CurrRec = 0;
      int NewPos = myFile.seek(sizeof(SampleRec) * CurrRec);
    }
    myFile.write((uint8_t *)&SampleRec, sizeof(SampleRec));
    myFile.close();
    myFile.flush();
    NextSequence++;
    CurrRec++;
    return true;
  } else return false;
}


happy coding.. ~q