reading and writing long int to a sd card

Hi everyone.

I could do with some advice on how to save unsigned long to an sd card and then read it back again when my arduino powers up.. Basically i am working on a project that uses a sensor to count the cycles of a machine at work. This works great and my project was advancing rapidly since i started to learn how to code from the middle of December 2019. Most of what i have learnt so far happened in the first 4 weeks, mostly learnt from this forum and from online videos.

Unfortunately i now need to ask for help. The problem i am having is understanding what i need to do to save just one, unsigned long to the sd everytime my sensor counts. I then need to read this number back into my variable to continue the counting the next day (restart of the next shift). I am completely confused after watching countless videos and reading on the forums for the last 4 weeks. atoi, atol, parseInt, char strings, char arrays, strings.

Could i please have some help and advice on how i need to resolve this problem so i can continue with my project and continue to learn more. This problem is holding me back.

Many thanks
Doug

Have you saved anything at all to the SD and retrieved it yet? What else have you tried?

The nice thing about SD cards is you can read and write them on your computer. So you can start writing files to your SD card and confirm on your computer. Once this all works you can start reading the data. I recommend you write your data into a text file rather then a binary format. This will allow you to see what you are doing using a text editor.

If you have any specific questions, please include more information about your system and libraries. e.g. which Arduino, what SD module add some links e.g. to datasheet, how you connected the module and which library you are using (including a link to the library e.g. on github if it is not from Arduino).

I would put the number into EEPROM in binary and just write it in ASCII to the file when it's changed.

That way you would not have to find, read and reconvert it from SD.

Hi thanks for getting back to me guys.

aarg I have been able to read and write the sd card using the the example file that comes with the arduino. Ive tried a few different bits of code that i copied from different forums or videos. None have been successful yet. The big problem is i dont really know what i am looking for. Im normally pretty good at working things out and solving problems

klaus I agree, i like the idea that if i need to i can take the sd card and view the files on a pc. This is not my current plan but something for the future as i learn more coding.
I'm currently using arduino uno R3. I'm using only the standard arduino SD.h & SPI.h libraries. The SD module is a Kwmobile Micro SD card reader and the card itself is a Sandisk Ultra sixteen GB MicroSDHC (sorry my number six key just conked out on me)

Whandall I did consider the eeprom as an option but decided not to as the amount of times i will write to it will be up to 10,000 times a day. I know that the eeprom has a limit as to how many times it can be written to.

So all i need to achieve at this stage is to save my variable to the sd card and read it back once when the arduino powers up.
This is how i think it will work.

My variable is a unsigned long called jobcount. Everytime the machine does one cycle it triggers the sensor which adds one to the count. The machine cycles about every 5.9 seconds. As the machine cycles and adds one to jobcount i wanted it to save it to the sd card. I would prefer to overwrite the file each time. At the end of the working day everything including the arduino will be powered off. Next morning everything powered up. Its at this point i figured in the set up code that the number saved on the sd card could be written/sent to jobcount variable ready to continue counting. Also saving the variable jobcount to sd is a backup in case of accidental power off.

I am i going about this the correct way or do i need to look for another way to save my variables.

Thanks
Doug

Try this:
how-do-i-write-integers-to-a-micro-sd-card-on-an-arduino

I thought SD cards also have a pretty low write limit? I have a similar issue where I need to store a count. Someone just recommended I2C Fram which looks promising for write cycles.

You can do wear leveling - append the number to the file instead of overwriting. To find the current value, just seek just before the end of the file because it will contain the latest value. Reading back could be slow if using sequential access instead of seek(), but you only have to do it once at boot time. To seek, find the length of the file and subtract 4 for the size of the last data item. Also delete the file and start it from scratch at every boot time after capturing the old value.

Thanks for the additional information. Here are the options I would consider in order of preference.

  1. Use a SPI FRAM chip. The technology has been around for a couple of decades but EEPROM has been the cheaper choice and therefore was used more. You could write your counter 10.000 times a day to it for much longer than any machine you can build. This will be the easiest choice for you.

ADAFRUIT SPI NON-VOLATILE FRAM BREAKOUT - 64KBIT

  1. Use a serial EERAM. Microchip has combined its EEPROM with RAM and automatic store/restore on power down and power up.

Microchip.com Serial-EERAM

  1. Use a EEPROM and only write to it when the power goes down. For this, you would need to detect when the power gets disconnected e.g. Vdd drops and then you get an interrupt, shut everything down and write your EEPROM with the remaining energy from a capacitor. This has been done for many years to get around the write count limits of EEPROMs.

  2. Use a EEPROM write only every 5 minutes or so and live with the counter error.

5 minutes = 100k/year = 10 years with a 1M E/W cycles endurance e.g. a Microchip 25LC010A

As suggested, use a wear leveling scheme for your eeprom writes, and place the data in different register locations as it is written. By moving the data around, you can greatly extend the life of the eeprom.

You want your count data as a long integer(32 bits), and consequently you will need to write the value to 4 bytes in the eeprom using EEPROM.put(). With each write, you will move this storage chunk progressively around in the eeprom to 256 possible locations. You can write 100K times 256 = 25,600,000.

A write every 6 seconds is 14400/day. 25,600,000/14400 = 1778 days or nearly 5 years. If you increment the count by 2 every 12 seconds you are looking at 9.75 years of eeprom life.

There will be a data value in eeprom bytes 0:3, then the next at bytes 4:7, then the next in 8:12 etc. When you get to the end of the eeprom space you start around again. When the power is on and the program is running the management of the next write location is straight forward.

When power is cycled, the program needs to find where the last written chunk was located, and to start writing again from that location.

Because your count data is monotonically increasing, the data becomes its own index to the last written location, and the wear leveling routine is pretty straightforward.

The way to find the last value written is to start reading the eeprom (using EEPROM.get()) from address 0 in 4 byte chunks and test each value against the next. When the value at an address is not bigger than the value in the previous address, you know that is where to write the next value.

You will need to start off with the eeprom at all 0's so that the comparison of each stored value to the next will work on the first pass. Additionally, if on a restart, you happen to find that all 512 locations contain increasing data values, then your next write will be at address 0.

To clear the eeprom to begin, set the boolean variable freshStart to true. Run the program. Then set freshStart = false, upload again, and your counting begins.

#include <EEPROM.h>
unsigned long count;
int index;
#define lastIndex  EEPROM.length() - sizeof(count) //1020
boolean freshStart = true;
//boolean freshStart = false;

void setup() {
  Serial.begin(115200);
  if (freshStart)
    clearEeprom();

  //find last count and index in Eeprom with checkEeprom()function
  Serial.print("Checking for last stored count = ");
  index = checkEeprom();
  Serial.println(EEPROM.get(index, count));
  Serial.print("Located at EEPROM index = ");
  Serial.println(index);
}

void loop() {
  //emulate machine cycle incrementing count and index
  static boolean machineCycleComplete = false;
  static unsigned long lastMachineComplete = 0;
  if (millis() - lastMachineComplete >= 2000)
  {
    machineCycleComplete = true;
    lastMachineComplete = millis();
  }
  //values to incremented by machine cycle
  if (machineCycleComplete == true)
  {
    machineCycleComplete = false;
    count += 2;
    index += sizeof(count);//increment address by 4 with long

    if (index > lastIndex)
      index = 0;

    EEPROM.put(index, count);
    Serial.print("index = ");
    Serial.print(index);
    Serial.print("  count = ");
    Serial.println(count);
    machineCycleComplete = false;
  }
}

void clearEeprom()
{
  Serial.println("Clear Eeprom Cells");
  for (int i = 0; i < EEPROM.length(); i++)
  {
    EEPROM.write(i, 0);
  }
  Serial.println("All eeprom cells set to zero.");
  Serial.println("Initialize freshStart to false and reload program");
  while (1);//stop program 
}

int checkEeprom()
{
  static unsigned long lastValue = 0;
  static unsigned long value = 0;
  for (int i = 0; i <= lastIndex; i += sizeof(count))
  {
    lastValue = value;
    EEPROM.get(i, value);
    if (value < lastValue)//next value is not larger than last
    {
        return (i - sizeof(count));//last valid index
    }
    if (i == lastIndex)//no smaller value found or values all equal as zeros
    {
      return i;
    }
  }
}

Hi all.

I'm finally writing and reading my jobcount to and from the sd card. The input from you all has been very helpful. In the end i manged to find a piece of code in another forum discussion. I used that on its own so i could test to see if it worked for me, and made a few adjustments in the process. As it was successful i then copied that code to my project code and then tweeked it again. Total success. I will now do the same for the other counts that it does, and work on other areas in my project. I dont fully understand how that section of code works but i know it will become clear as i mess around with it more. Im a visual learner so i have to try and picture what the code is doing and when.

Cattledog - That was a really in-depth description you wrote there and the piece of code will be very useful for me to try/use. Its out of my depth at the moment but i will try it in the coming weeks.

Every option discussed in this topic will be tried in time.

Doug

2 Likes