EEPROM Storage issue

Guys,
I know it's a damn stupid question. But still need your kind support as i'm going crazy on finding exact bugs...

[EDITED]

  1. I need to store any integer signed value from -50 to 50 (signed integer).
  2. I need to store just 1 or 0 (unsigned integer, actually boolean)
  3. I need to store one digit from 1 to 9 (unsigned int, single digit)
  4. I need to store any single digit from -9 to 9 (signed single digit integer)
  5. I need to store a 10 digit value (randomly generated 10 digit value char or integer, unsigned)
  6. I need to store a 4 digit value (say 4 digit PIN, char/integer, unsinged)
  7. I need to store a 4 digit string (char string)

Now, for each case, I need to know how many bytes it may consume (sure that's not a big deal to find). But actually i more importantly need to know, how many eeprom address blocks will be needed for each case.

I suggest that you look at the EEPROM reference for the get() and put() functions.

1-4 1 byte (char or byte).
5-7 how is that stored? As a string?

The strlen() function for the number of characters (bytes) in a string (not the amount of memory allocated for the string).

struct eepromCrap {
  char anyValue50to50;// char is 8 bits, holds -128 to 127
  bool just1or0;// bool is 8 bits holds TRUE or FALSE
  unsigned char oneDigit;// unsigned char is 8 bits holds 0-255
  char anyValue9to9;// signed char is 8 bits -128 to 127
  unsigned long random10digit;// unsigned long is 32 bits holds 0 to 4,294,967,295
  unsigned int number4DigitPIN;// unsigned int is 16 bits 0 - 65,535
  char myString[5];// 40 bits since it holds the 4 characters and a NULL
} myEEprom;

size_t x = sizeof(myEEprom);
Serial.print(x);// should be 15 bytes

groundFungus:
I suggest that you look at the EEPROM reference for the get() and put() functions.

1-4 1 byte (char or byte).
5-7 how is that stored? As a string?

The strlen() function for the number of characters (bytes) in a string (not the amount of memory allocated for the string).

made small correction for your better understanding

for everyone's ease,
I can find the size from codes for sure... but i need to know for this type of data, how many eeprom blocks I need for each case. like to store a boolean, only 1 eeprom block is enough (actually overkill). For a 0-254 still i can use a single eeprom block or -127 to +127 single eeprom block and eeptomreadint() is enough. But what if it's a bit more... say 10 digits? or what if i need to store 0000 to 9999 (I prefer char as 0000 is here a value to, a PIN/password.)

How are the data stored? A 10 digit number* stored as a string takes 11 bytes (10 digits plus terminating null).
0000 to 9999 for a password best stored as a string (my opinion) so 4 bytes + 1 for null.

I don't know how you would store a 10 digit number as a number. Long (4 bytes) can hold -2,147,483,648 to 2,147,483,647, unsigned long can hold 0 to 4,294,967,295 but outside those ranges I don't know.

For even more better understanding, here is my EEPROM MAP: (Surely buggy)
Please read the contents in { } as my actual what will be stored

//EEPROPM Addressings need to be re-check for more optimization
#define eepromOffset 0 //[Address 0, reserved]
#define deviceSLAddr (eepromOffset + 1) //[Start at 1, used for device SL (10byte incl NULL), ends at 11] {Will store a 10 digit Serial Number, eg. 1421256398}
#define optHWEn_eepromAddr (deviceSLAddr + 11) //[Start at 1+11=12, used for 1byte, ends at 12] {will store either 0 or 1}
#define debugEn_eepromAddr (optHWEn_eepromAddr + 1) //[Start at 12+1=13, used for 1byte, ends at 13] {will store either 0 or 1}
#define tempSet_eepromAddr (debugEn_eepromAddr + 1) //[Start at 13+1=14, used for 2byte, ends at 15] {will store any signed integer from -100 to +100}
#define deltaT_eepromAddr (tempSet_eepromAddr + 2) //[Start at 15+2=17, used for 1byte, ends at 17] {will store any singed integer from -9 to +9}
#define opsMode_eepromAddr (deltaT_eepromAddr + 1) //[Start at 16+1=17, used for 1byte, ends at 17] {will store unsinged int from 1 to 3}
#define compressorRunTime_eepromAddr (opsMode_eepromAddr + 1) //[Start at 17+1=18, used for 1byte, ends at 18] {will store unsigned integer of 2 digit. From 1 to 99}
#define defrostTime_eepromAddr (compressorRunTime_eepromAddr + 1) //[Start at 18+1=19, used for 1byte, ends at 19] {will store unsigned integer of 2 digit. From 1 to 99}
#define userPassword_eepromAddr (defrostTime_eepromAddr + 1) //[Start at 19+1=20, used for 4byte, ends at 23] {will store 4 chars, e.g. 1111 }
#define deviceSLlength 10 //10 digit including NULL char [0-9 total 10 + 1]
#define userPasswordLength 5 //4 char incl NULL (dont know why 4 is not working) (eeprom write int is taking 2 extra may be)

char devSL[deviceSLlength]; //10 digit device SL No including NULL char [0-9 total 10 + 1]

// The necessary variables for regular operation
boolean optHWEn = false;
boolean debugEn = false;
unsigned int opsMode = 1; // default ops mode = 1
unsigned int deltaT;
char userPass[userPasswordLength] = "1111"; //default user pass = 1111
unsigned long compressorRunTime = 0;
unsigned long defrostTime = 0;
int targetTemp;

Now hope this will help

actual culprit:

#define userPasswordLength 5 //4 char incl NULL (dont know why 4 is not working) (eepromWriteString is taking 2 extra may be)

in my eyes!!!

It is impossible to tell from your EEPROM MAP whether the number of bytes you have allocated for each variable is correct because we don't know what data types they are

Unless you intend to store more data in the EEPROM you have plenty of room so why not save each variable at say 20 byte intervals. Storing them crammed together or spaced apart will not affect performance

(eepromWriteString is taking 2 extra may be)

What is eepromWriteString ? It is not mentioned in the code that you have posted. Please post a complete sketch

aq_mishu:
for everyone's ease,
I can find the size from codes for sure... but i need to know for this type of data, how many eeprom blocks I need for each case. like to store a boolean, only 1 eeprom block is enough (actually overkill). For a 0-254 still i can use a single eeprom block or -127 to +127 single eeprom block and eeptomreadint() is enough. But what if it's a bit more... say 10 digits? or what if i need to store 0000 to 9999 (I prefer char as 0000 is here a value to, a PIN/password.)

one of these days I will get around to writing an actual eeprom tutorial, because I think people just don't get it.... you HAVE to move from EEPROM to RAM in order to use eeprom, and it's likely you will do that in setup... here are some basic steps... 1) organize EEPROM data into a data structure (hence my example earlier of a struct)... this takes all the stress out of addresses, as you simply refer to data structure members (e.g. myEEprom.anyValue9to9) 2) code a function to read from eeprom to ram using the pointer, byte by byt using EEPROM.read(), indexing the address by 1 (i.e. one byte) in a for loop until you have reached sizeof(myEEProm) less 1, because it's going to be pointer+0 is the first address, and pointer+(sizeof(myEEprom)-1) is the last address.. 3) write a function that writes to EEprom, using EEPROM.update() in a similar fashion.... but reverse your pointers to and from....

a function to read from eeprom to ram using the pointer, byte by byt using EEPROM.read()

Or do it the easy way and use EEPROM.get() and EEPROMN.put() to load/save the whole struct at the same time with a single command

Use long for everything because eventually you will overwrite your data with another sketch. By using longs it will be easier to see which fields are ruined. Imho.

in my post #6, the the contents in { } will be kind of as my actual what will be stored. About data types, and max/min data. That's why i was keeping i fixed length. (Though could keep margin).

Yep, the struct is a good way... I will post more soon.

MY variables are:

#define deviceSLlength 10 //10 digit including NULL char [0-9 total 10 + 1]
#define userPasswordLength 5 //4 char incl NULL (dont know why 4 is not working) (eeprom write int is taking 2 extra may be)
char devSL[deviceSLlength]; //10 digit device SL No including NULL char [0-9 total 10 + 1]
// The necessary variables for regular operation
boolean optHWEn = false;
boolean debugEn = false;
unsigned int opsMode = 1; // default ops mode = 1
unsigned int deltaT;
char userPass[userPasswordLength] = "1111"; //default user pass = 1111
unsigned long compressorRunTime = 0;
unsigned long defrostTime = 0;
int targetTemp;

My initial eeprom storage as factory loaded eeprom data (I mean my pre-loaded data):

//Generate device Serial number
void generate_devSL(unsigned int a){
    debug.println(F("Generating random Serial since first boot."));
    for (int i = 0; i < a; i++) {
      devSL[i] = TrueRandom.random(49, 58);
      debug.print(devSL[i]);
      EEPROM.update(i+1, devSL[i]);
      delay(100);
    }
    EEPROM.update(0, '#');
    debug.println();
    factoryReset();
    digitalWrite(systemStateLED, LOW);
}

//The factory reset function that will reset EEROM data into default.
void factoryReset(void){
  debug.println(F("Resetting all values to factory default..."));
  digitalWrite(systemStateLED, HIGH);
  EEPROM.update(optHWEn_eepromAddr, 1);
  delay(5);
  EEPROM.update(debugEn_eepromAddr, 1);
  delay(5);
  EEPROMWriteInt(tempSet_eepromAddr, 4);
  delay(5);
  EEPROMWriteInt(deltaT_eepromAddr, 4);
  delay(5);
  EEPROM.update(opsMode_eepromAddr, 1);
  delay(5);
  EEPROM.update(compressorRunTime_eepromAddr, 12);
  delay(5);
  EEPROM.update(defrostTime_eepromAddr, 30);
  delay(5);
  userPass[userPasswordLength] = "1111";
  eeprom_write_string(userPassword_eepromAddr, userPass);
  delay(5);
  digitalWrite(systemStateLED, LOW);
  debug.println(F("Done!"));
  debug.println();
  loadBootConf();
  printBootConf();
  debug.print(F("Default User Password: ")); debug.println(userPass);
}

//Function for loading the conf during boot. Also the same function is used to call the runtime conf after a new conf settings changed.
void loadBootConf(void){
  
  optHWEn = EEPROM.read(optHWEn_eepromAddr);
  debugEn = EEPROM.read(debugEn_eepromAddr);
  targetTemp = EEPROMReadInt(tempSet_eepromAddr);
  deltaT = EEPROM.read(deltaT_eepromAddr);
  opsMode = EEPROM.read(opsMode_eepromAddr);
  compressorRunTime = (EEPROM.read(compressorRunTime_eepromAddr) * 3600000); //eeprom value in hours. Hence 3600000 ms
  defrostTime = (EEPROM.read(defrostTime_eepromAddr) * 60000); //eeprom value in minutes, hence 60000 ms
  eeprom_read_string(userPassword_eepromAddr, userPass, sizeof(userPass)+1);
}

//A fancy way of showing the running conf.
void printBootConf(void) {
  debug.println(F("Boot configuration:"));
  debug.print(F("Optional Hardwares Enable: ")); debug.println(optHWEn);
  debug.print(F("Debug Enable: ")); debug.println(debugEn);
  debug.print(F("Target Temp: ")); debug.print(targetTemp); debug.println(F("C"));
  debug.print(F("Delta T: ")); debug.println(deltaT);
  debug.print(F("Ops Mode: ")); debug.println(opsMode);
  debug.print(F("Compressor Run Time: ")); debug.print(compressorRunTime/3600000); debug.println(F(" Hrs"));
  debug.print(F("Defrost Time: ")); debug.print(defrostTime/60000); debug.println(F(" mins"));
  debug.println();
  debug.print(F("Low cut at: ")); debug.print(lowTempLimit); debug.println(F("C"));
  debug.print(F("Hi cut at: ")); debug.print(hiTempLimit); debug.println(F("C"));
}

Perehama:
char anyValue50to50;// char is 8 bits, holds -128 to 127

...except when it doesn't.

aq_mishu:
...
delay(100);
...
delay(5);
...
delay(5);
...

What's the deal with all those delays?

/*
 * [ SECTION: 1 ]
 * 
 * This section will cover extended EEPROM functions.
 * 
 */
//This function will write a 2 byte integer to the eeprom at the specified address and address + 1
void EEPROMWriteInt(int p_address, int p_value) {
  byte lowByte = ((p_value >> 0) & 0xFF);
  byte highByte = ((p_value >> 8) & 0xFF);
  EEPROM.update(p_address, lowByte);
  EEPROM.update(p_address + 1, highByte);
}

//This function will read a 2 byte integer from the eeprom at the specified address and address + 1
unsigned int EEPROMReadInt(int p_address) {
  byte lowByte = EEPROM.read(p_address);
  byte highByte = EEPROM.read(p_address + 1);
  return ((lowByte << 0) & 0xFF) + ((highByte << 8) & 0xFF00);
}

//This function will write a 4 byte (32bit) long to the eeprom at
//the specified address to address + 3.
unsigned long EEPROMWriteLong(int address, long value) {
  //Decomposition from a long to 4 bytes by using bitshift.
  //One = Most significant -> Four = Least significant byte
  byte four = (value & 0xFF);
  byte three = ((value >> 8) & 0xFF);
  byte two = ((value >> 16) & 0xFF);
  byte one = ((value >> 24) & 0xFF);

  //Write the 4 bytes into the eeprom memory.
  EEPROM.write(address, four);
  EEPROM.write(address + 1, three);
  EEPROM.write(address + 2, two);
  EEPROM.write(address + 3, one);
}

unsigned long EEPROMReadLong(long address){
  //Read the 4 bytes from the eeprom memory.
  long four = EEPROM.read(address);
  long three = EEPROM.read(address + 1);
  long two = EEPROM.read(address + 2);
  long one = EEPROM.read(address + 3);

  //Return the recomposed long by using bitshift.
  return ((four << 0) & 0xFF) + ((three << 8) & 0xFFFF) + ((two << 16) & 0xFFFFFF) + ((one << 24) & 0xFFFFFFFF);
}

// Returns true if the address is between the
// minimum and maximum allowed values, false otherwise.
//
// This function is used by the other, higher-level functions
// to prevent bugs and runtime errors due to invalid addresses.
boolean eeprom_is_addr_ok(int addr, int buffSize, int maxBuffSize) {
  return ((addr + buffSize <= maxBuffSize));
  //return ((addr >= EEPROM_MIN_ADDR) && (addr <= EEPROM_MAX_ADDR));
}

// Writes a sequence of bytes to eeprom starting at the specified address.
// Returns true if the whole array is successfully written.
// Returns false if the start or end addresses aren't between
// the minimum and maximum allowed values.
// When returning false, nothing gets written to eeprom.
boolean eeprom_write_bytes(int startAddr, const byte* array, int numBytes) {
  // counter
  int i;
  int EEPROM_MAX_ADDR = (startAddr + 99); //This is the total byte (ADDR) allocation for one string. 99 bytes including the given one (addr) + 1byte NULL Terminator.
  // both first byte and last byte addresses must fall within
  // the allowed range 
  //if (!eeprom_is_addr_ok(startAddr) || !eeprom_is_addr_ok(startAddr + numBytes)) {
    //return false;
  //}
  if (!eeprom_is_addr_ok(startAddr, numBytes, EEPROM_MAX_ADDR)) { // check start address
    return false;
  }
  for (i = 0; i < numBytes; i++) {
    EEPROM.write(startAddr + i, array[i]);
  }
  return true;
}

// Writes a string starting at the specified address.
// Returns true if the whole string is successfully written.
// Returns false if the address of one or more bytes fall outside the allowed range.
// If false is returned, nothing gets written to the eeprom.
boolean eeprom_write_string(int addr, const char* string) {
  int numBytes; // actual number of bytes to be written
  //write the string contents plus the string terminator byte (0x00)
  numBytes = strlen(string) + 1;
  return eeprom_write_bytes(addr, (const byte*)string, numBytes);
}

// Reads a string starting from the specified address.
// Returns true if at least one byte (even only the string terminator one) is read.
// Returns false if the start address falls outside the allowed range or declare buffer size is zero.
// 
// The reading might stop for several reasons:
// - no more space in the provided buffer
// - last eeprom address reached
// - string terminator byte (0x00) encountered.
boolean eeprom_read_string(int addr, char* buffer, int buffSize) {
  int EEPROM_MAX_ADDR = (addr + 99); //This is the total byte (ADDR) allocation for one string. 99 bytes including the given one (addr) + 1byte NULL Terminator.
  byte ch; // byte read from eeprom
  int bytesRead; // number of bytes read so far
  if (!eeprom_is_addr_ok(addr, buffSize, EEPROM_MAX_ADDR)) { // check start address
    return false;
  }
  if (buffSize == 0) { // how can we store bytes in an empty buffer ?
    return false;
  }
  // is there is room for the string terminator only, no reason to go further
  if (buffSize == 1) {
    buffer[0] = 0;
    return true;
  }
  bytesRead = 0; // initialize byte counter
  ch = EEPROM.read(addr + bytesRead); // read next byte from eeprom
  buffer[bytesRead] = ch; // store it into the user buffer
  bytesRead++; // increment byte counter
  // stop conditions:
  // - the character just read is the string terminator one (0x00)
  // - we have filled the user buffer
  // - we have reached the last eeprom address
  while ( (ch != 0x00) && (bytesRead < buffSize) && ((addr + bytesRead) <= EEPROM_MAX_ADDR) ) {
    // if no stop condition is met, read the next byte from eeprom
    ch = EEPROM.read(addr + bytesRead);
    buffer[bytesRead] = ch; // store it into the user buffer
    bytesRead++; // increment byte counter
  }
  // make sure the user buffer has a string terminator, (0x00) as its last byte
  if ((ch != 0x00) && (bytesRead >= 1)) {
    buffer[bytesRead - 1] = 0;
  }
  return true;
}

void factoryEEPROMFlush(unsigned int memLimit){
  debug.print(F("Flushing 0 to ")); Serial.print(memLimit);Serial.println(F(" address blocks with 0."));
  digitalWrite(13, HIGH);
  for (int i=0; i <= memLimit; i++){
    EEPROM.write(0, i);
    delay(5);
  }
  digitalWrite(13, LOW);
  debug.print(F("Flushed ")); Serial.print(memLimit);Serial.println(F(" addresses"));
  debug.println(F("Please do a power cycle."));
}

Above is the eeprom extended functions...

void EEPROMWriteInt(int p_address, int p_value) {
  byte lowByte = ((p_value >> 0) & 0xFF);
  byte highByte = ((p_value >> 8) & 0xFF);
  EEPROM.update(p_address, lowByte);
  EEPROM.update(p_address + 1, highByte);
}

Why not use EEPROM.put() and EEPROM.get() instead of messing around with spitting an int into bytes and writing/reading them individually ? The put() function automatically uses update() behind the scenes so will not unnecessarily add to the number or writes to the EEPROM

so to store say int i=26 it should be eeprom.put(26, ADDR of storage) right?

Now, if i need to store int i=26 and int j=36 then if the starting eeprom addr for i is 1, then what should be for j??

[actually that's the first issue here]

The syntax of put() is

EEPROM.put(address, data)

You specify the address and data can either be a value or a variable

Why not use a struct ?

#include <EEPROM.h>

struct dataLayout
{
  int i;
  int j;
  char * name;
};
dataLayout data1;
dataLayout data2;

void setup()
{
  Serial.begin(115200);
  while (!Serial);
  data1.i = 26;
  data1.j = 36;
  data1.name = "Bob";
  EEPROM.put(0, data1);  //put the data into EEPROM
  EEPROM.get(0, data2);  //get the data back from EEPROM
  Serial.println(data2.i);
  Serial.println(data2.j);
  Serial.println(data2.name);
}

void loop()
{
}

This is just an example to prove that it works. Obviously in practice you only need one struct. You can add and remove data items, putting them in any order you like as long as you do a put() before the next get(). There is no need to calculate where to save each byte, just save the whole struct

the structure method is a nice one i do fully agree, but just for the sake of debate here:

I will store several runtime parameters and i may change anyone of those, say debugEnable and then it needs to UPDATE only that section, instead of the whole thing. That was actually my motivation for individual addressing..

I will add another code block soon to let you know what i'm doing