SD card read/write with Arduino

Sorry, Triffid, but as I said the ATmega is capable of self-programming, so yes it is technically possible for it to pull code from the SD card, write it to flash, and execute it. This really isn't any different conceptually from pulling the data from a serial port and writing it to flash, which is exactly what the Arduino bootloader does for us, allowing us to program the ATmega without a hardware programmer.

As I said earlier I don't think it's terribly practical, but it absolutely is possible.

-j

So back on topic... SD mode IO? SDHC support?

ducttapepro6---

I do also experience som trouble after resetting Arduino. Everything works fine to read and write to the SD Card but after resetting the Arduino the SD Card data is blank.

Have you found any solution for this?

//Daniel

Zumbi, I am having the same problem as you. I cannot get the SD card to initialize. If anyone has some pointers I would be grateful. : ::slight_smile:

Is it possible to change the Arduino pins that are used by the library? I can't seem to find any reference to the pin numbers in the library source code.

Thom

EDIT

I've found out you need to use the SPI pins, so cannot change the pins in use by the SD card reader... time to get rewiring :smiley:

Hi everyone,

Earlier, ducttapepro6 wrote:

Mr. Orange-

I had no trouble getting your code up and running in 15 minutes - happily logging, saving, and reading the input to analog1.

However, after hitting the arduino reset button, I found that the read function returns junk(from the SD card).

Shouldn't the SD card retain data after a power cycle? or, is it a function of the code that you so nicely supplied to get me started - and I just don't understand it, yet?

Along with Daniel, I'm hoping that someone can explain why the card starts up with the same junk data each time, if the first thing I do when the sketch runs is hit 'r'.

agent_orange says that it's like having a 1G (or whatever) EEPROM, which is IDEAL for my situation. My arduino will be sitting in a box, on top of a pole, powered by a solar panel for up to 3 months at a time. I have it successfully writing the date and max & min temperatures to the EEPROM, which I can read back out as desired. The EEPROM data is only overwritten after a new sketch is started. This is the functionality that I require from the SD card, but the supplied code appears to wipe the card each time the sketch is started. (I tried commenting out the if(!sd_raw_init()) but that didn't help).

agent_orange, thanks so much for supplying this work, it's fantastic & there's no WAY that I'd be able to work this out on my own. But if anyone can show me a way (even if it's a seperate sketch) to access the data on the card without destroying it, it would be GREATLY appreciated!!

I guess the other option would be if someone could recommend a Windows-based program which could access the data on the SD card?

Thanks for reading everyone!!

I am using the same simple logging code that you have posted with just the raw read and write. The initialization seems to go fine, and then it acts like it is reading and writing even when i have the SD card removed from the slot. I enter the 'r' command, and it prints the same info, not zeros or junk. when i reset, it loses the info. It is as if it is just storing it in on board memory until it resets, like it isn't even using the sd card at all, and then loses the data when it resets.

Any ideas?

PS...
Thank you Agent Orange for working on this project.

Same here... at first it looks like it's working fine, it appears I can sample data to the card and read it back. But once I've reset the board and try and read the data back I am presented with all zero's. :frowning:

I too can confirm that reading and writing appears to work even without a card in the slot, despite the fact that it says initialisation failed and doesn't show any card statistics. :-/

With a card in the slot I get reasonable looking card statistics (size: 125698048 for a 128MB card) which suggests it can see the card.

Which again raises the question, what are we missing here? Any ideas?

I will be sure to post if I make any further discoveries.

Thanks all! :slight_smile:

Hi there.

I found another solution that I think is much better for storing large amount of data and it is Fat16+32 compatible. You can communicate with the card with normal serial, read more: http://www.roguerobotics.com/ uMMC datastorage.

Regards /Daniel :slight_smile:

Hi Daniel,

That's certainly a nice-looking solution! Might be a bit of overkill for me though; using a 16Mb SD card that came with my camera, I worked out that if I store 10 bytes of data every minute, I've got over 2.5 YEARS worth of storage capacity...

Just for my curiosity, how would you communicate with this uMMC device? I'm guessing that you'd have to Serial.print() commands like "O 1 R /LOGS/JANUARY/JAN03.LOG{cr}" to it in it's own language?

Thanks!

Okay, I am getting somewhere...

I thought I would try the uFat library that was mentioned fairly early on in this thread, which straightaway gave me a directory listing of all the files on this card. But hang on a minute, I thought I had formatted the card! So I popped it back into the computer, deleted all the files and created the required data.bin file, but to my amazement my new file disappeared and all the old ones came back! No amount of deleting formatting or mortifying seemed to affect this card any more, so I have scrapped it and got myself a new one.

uFat seemed very temperamental with my new card, failing to initialise nine out of 10 times. I put a simple loop round the initialising statement with a delay, so it would retry if it failed instead of just showing an error and halting. This seemed to work a treat, no matter how much delay before or after it always seems to fail once and then initialise fine on the second attempt without fail. Reading and writing then worked perfectly :smiley:

I think I am going to stick with uFat over the raw library as it isnt any bigger and I still can't get the raw library to work even with my new card (same problems as described before), also I like being able to read my data back on the computer. :slight_smile:

The only downside is that the code for using uFat isn't as friendly as the raw library, and I can't seem to get it to write one byte at a time, it has to be written in 512byte sectors, but I can live with this.

Hopefully this information helps someone out, I am thinking to get the raw library working perhaps something needs to be inside a retry loop much like I had to do with the uFat library to get it working.

Thanks everyone who has contributed, especially agent_orange for the circuit diagram (which needs no modification to work with uFat), and sirmorris for the uFat library.

Hey bobemoe this sounds good. I've tried the RAW program and haven't had any luck. I'm currently attepmting SirMorris's code and am not getting anywhere there either. I get some errors when I try to compile.

o: In function microfat::initialize(unsigned char*)': multiple definition of microfat::initialize(unsigned char*)'hardware\libraries\uFat\microfat.o:C:\arduino-0010/hardware\libraries\uFat\microfat.cpp:17: first defined here

o: In function microfat::locateFileStart(char const*, unsigned long&, unsigned long&)': o: In function main':

Any sugggestions?
I have the required header files and I'm trying to run the microfat program.

My apologies if this has already been resolved somewhere else :sunglasses:

Hi smelios,

It doesn't look like you are using the latest version, try downloading uFat2.zip rather than mmc1.rar, hopefully that will fix the problem.

Fresh code mmmmmm.

Still having problems. Bobemoe, did you encounter an error like this?

error: 'directory_entry_t' was not declared in this scope In function 'bool showDirectory_walkerfn(directory_entry_t*, unsigned int, void*)':
In function 'void showDirectory()':

Or did yours compile with no problems?

Sorry to keep asking you questions i'm a noob. ;D
Your help is greatly appreciated!

Mine compiled fine.

I can get a similar error if i remove the #include lines from the top of the example which makes me think its a problem with your includes.

Did you delete the files you copied to /hardware/libraries/ for version 1? they may be conflicting.

Found my error, I was on arduino10. Morris said he developed this in Arduino12 so i downloaded that and it complied. Now i just can't init the sd card. Oh well work for another day :stuck_out_tongue:
Thanks again for helping

Excellent, now you are at the same stage that I was. Try making it retry the initialisation. This is what fixed it for me, I changed:

  if (mmc::initialize() != RES_OK)
  {
    error(PSTR("mmc init failed.\n"));
  }

Changed to:

  while(mmc::initialize() != RES_OK){
    pprint(PSTR("mmc init failed. retrying...\n"));
    delay(100);
  }

I find it always fails once. If that doesn't fix it I am all out of ideas. Good luck.

Hi again,

After further reading (ie that uFat basically takes up the same space as the sd_raw instructions) I've headed down the uFat track. I've downloaded the uFat2 files from SirMorris's blog (http://groups.google.com/group/micro-hacker
) and have integrated them into my program, although I have a few problems...

First, the sketch seems to 'interfere' with my DS1307 real time clock data. With the SD card connected as per the diagram at the beginning of this thread (and the clock connected to analog pins 4 & 5), the clock output is 'mixed up', e.g. the day comes through as the hour, year comes through as day of the month, etc. Leaving the hardware connected and loading up a sketch which ONLY displays output from the RTC, I get the correct output from the clock.

Secondly, I'm not too sure as to exactly how the reads/writes work. Am I correct in thinking that if (RES_OK == mmc::writeSectors(sector_buffer, sector, 1)) simply checks that a 1-byte write to sector (which=0) of sector_buffer works? Is it just the digitalWrite(13, (millis() / 1000) & 1) which writes sector_buffer out to the SD card? If that's correct, can anyone explain how this works for me? I'm very much learning as I go...

I'd like to be able to write data out to the SD card 10 bytes at a time (is this possible, or does the buffer have to be 512 bytes?), and be able to read them back & display them when prompted. Is this also possible, or am I only going to be able to check the output by checking the data.bin file on the PC?

Thanks again for everyone's thoughts...

JB

Hi JB.

The line:

if (RES_OK == mmc::writeSectors(sector_buffer, sector, 1))

writes 1 sector's worth of data stored at sector_buffer in the arduino's SRAM to the specified sector on the card. Once this routine returns your data is stored. The line:

digitalWrite(13, (millis() / 1000) & 1)

has nothing to do with the workings of the program. It simply flashes the LED on port 13 to let you know all's done.

All reads and writes in uFat are 512 bytes. This is the size of a sector on the card. It would be possible to write a function that allows arbitrarily-sized writes but that would add to the size.

Do you only want to ever have 10 bytes stored? Or are you saving a stream? When you want to read it back would you be showing all the data stored thus far? What's the expected data rate? If it's a slow dribble of data to the card that would warrant a different approach than if you're going for realish-time.

If you can describe your program requirements I'll see wht I can do to help you out.

There's no obvious reason why uFat should interfere with your RTC. It sounds like you're using 2wire for the clock. I have a board that uses I2C components simultaneously with hardware SPI without a problem, I may need to look over your code.

Charlie

Hi Charlie,

Thanks for clarifying that for me, I think I get it now.

I think I can probably work with the 512 byte write each time... My program is monitoring a temperature reading and the voltages of a battery and a solar panel, in an enclosure housing a camera to take a few photos each day, producing 10 bytes of data per minute. I could probably drop this down to 8 bytes if writing 10 bytes/min causes problems with the 512 byte sector size, but I'd prefer not to.

Here's my code (over 2 posts):

#include <Wire.h>
#include <math.h>
#include <DateTime.h>
#include <avr/pgmspace.h>
#include <wprogram.h>
#include <microfat2.h>
#include <mmc.h>

#define ThermistorPIN 0            // input read pin for LM335 is Analog Pin 0
float temperature = 0;                 // variable which will be calculated in process
long val=0;                       // variable to store the value coming from the thermistor
float temp;

unsigned long sector = 0;
unsigned long byteSize;

unsigned long WriteAddress = 0;    // initial address for writing to the SD card.
byte value;                        // will be used...
int incomingByte = 0;                 // ...for incoming serial data
byte tempBytes[10];                // for sending data to the SD card
byte sector_buffer[512];           // for sending AND retrieving data to/from the SD card

int hour;
int minute;
int second;
int month;
int day_of_week;
int day;
int year;

char* dow[7] = {"Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"};

float Thermistor(int RawADC) {  //  Inputs ADC Value from Thermistor and outputs Temperature in Celsius
 long Resistance;
 float Temp;                                      // Dual-Purpose variable to save space.
 Resistance=((10240000/RawADC) - 10000);          // Assuming a 10k Thermistor.  Calculation is actually: Resistance = (1024/ADC)
 Temp = log(Resistance);                          // Saving the Log(resistance) so not to calculate it 4 times later. "Temp" means "Temporary" on this line.
 Temp = 1 / (0.001129148 + (0.000234125 * Temp) + (0.0000000876741 * Temp * Temp * Temp));   // Now it means both "Temporary" and "Temperature"
 Temp = Temp - 273.15;  // Convert Kelvin to Celsius                                            Now it only means "Temperature"
 return Temp;           // Return the Temperature
}

void print_prompt_P(const char *data) {         // print prompts and messages from program flash to save SRAM
   while (pgm_read_byte(data) != 0x00)
     Serial.print(pgm_read_byte(data++));
}

void TakePhoto() {
  digitalWrite(3,HIGH); // first, switch on the fans to blow away dust or bugs...
  delay(10000);
  digitalWrite(3,LOW);
  delay(2000);
  digitalWrite(9,HIGH); // then send power to the camera
  delay(500);           // time for current to get to camera - not sure if this is needed...
  digitalWrite(6,HIGH); // trigger the solenoid for 100 msec
  delay(100);
  digitalWrite(6,LOW);
  delay(15000);         // wait 15 seconds for the camera to take a photo then shutdown
  digitalWrite(9,LOW);  // stop sending power to the camera.
}

void DataDump () {
  int DaysToDump = WriteAddress;
  Serial.println("");
  print_prompt_P(PSTR("          Data dump initiated: dumping "));
  Serial.print(DaysToDump/10);
  print_prompt_P(PSTR(" days of data.\n"));
    print_prompt_P(PSTR("Date,Time,Temp,VBatt,VPanel\n"));
      for(int p=0;p<DaysToDump;p=p+10) {
        sector_buffer[p]=digitalRead(13);
        Serial.print(sector_buffer[p]);

        Serial.print(analogRead(0));  // this bit dumps the date in 'dd/mm/yy' format...
          print_prompt_P(PSTR("/"));
//          ++f;
          Serial.print(analogRead(0));
          print_prompt_P(PSTR("/"));
          delay(50);
          Serial.print(analogRead(0));
          print_prompt_P(PSTR(","));
          Serial.print(analogRead(0));  // this bit prints the time, as ',HH:MM,'...
          print_prompt_P(PSTR(":"));
//          ++f;
          Serial.print(analogRead(0));
          print_prompt_P(PSTR(","));
//          ++f;
          delay(50);
          Serial.print(analogRead(0));  // this bit prints the temperature...
          print_prompt_P(PSTR(","));
//          ++f;
          delay(50);
          Serial.print(analogRead(0));  // this bit prints the Battery Voltage in 'VV.vv,' format...
          print_prompt_P(PSTR("."));
//          ++f;
          Serial.print(analogRead(0));
          print_prompt_P(PSTR(","));
//          ++f;
          delay(50);
          Serial.print(analogRead(0));  // this bit prints the Solar Panel Voltage in 'VV.vv,' format...
          print_prompt_P(PSTR("."));
//          ++f;
          Serial.print(analogRead(0));
//          ++f;
          delay(50);
        Serial.println();
      }
  WriteAddress = 0;
}

void error(const char* s)
{
  print_prompt_P(PSTR("Error: "));
  print_prompt_P(s);
  print_prompt_P(PSTR("<press reset>\n"));
  for( /* ever */ ; ; ) 
  {
    digitalWrite(13, (millis() / 250) & 1);
  }
}

void SDWrite() {
  // Pass in the sector-sized buffer we'll be using ourselves later.
  // uFat doesn't own it, it just needs to use it temporarily.
  // We also pass in the address of a function that is used to read
  // sectors from our device.
  //  
  if (!microfat2::initialize(sector_buffer, &mmc::readSectors))
  {
  error(PSTR("uFat init failed.\n"));
  }
  prog_char string_0[] PROGMEM = "DATA    BIN";

  if (microfat2::locateFileStart(PSTR("DATA    BIN"), sector, byteSize)) // check if the target file exists
  {
    if (byteSize >= 512)                                                 // check it's bigger than 512 bytes
    {
      if (RES_OK == mmc::readSectors(sector_buffer, sector, 1))          // checks a single-SECTOR read works OK
      {
          print_prompt_P(PSTR("Data read test SUCCESSFUL.\n"));
          delay(500);
        if (RES_OK == mmc::writeSectors(sector_buffer, sector, 1))       // checks a single-SECTOR write works OK
        {
          print_prompt_P(PSTR("Data write test SUCCESSFUL.\n"));
          delay(500);
        }
        else
        {
          print_prompt_P(PSTR("Failed to write updated data."));         // if the write fails, end here...?
        }
      }
      else
      {
        print_prompt_P(PSTR("Failed to read data.bin."));
      }
    }
    else
    {
      error(PSTR("Found data.bin, but it's too small."));
    }
  }
  else
  {
    print_prompt_P(PSTR("data.bin not present on card."));
  }
  Serial.println("Initialisation COMPLETE!!");
}

void setup() {

 pinMode(3,OUTPUT);       // FANS control - YELLOW wire
 digitalWrite(3,LOW);
 pinMode(6,OUTPUT);       // SOLENOID power control - WHITE wire
 digitalWrite(6,LOW);
 pinMode(9,OUTPUT);       // CAMERA power control - BLUE wire
 digitalWrite(9,LOW);
// pinMode(13,OUTPUT);
 Serial.begin(115200);
 Wire.begin();

// The rest of this setup() procedure is checking that the SD card is present & working.

  while(mmc::initialize() != RES_OK)
  {
    print_prompt_P(PSTR("SD init failed. retrying...\n"));
    delay(100);
  }
  SDWrite;
}

void loop() {

 // Below required to reset the register address to 0.
 Wire.beginTransmission(104); // transmit to device #104, the ds 1307
 Wire.send(0x00);
 Wire.endTransmission();      // stop transmitting

 Wire.requestFrom(104, 7);    // request 7 bytes from slave ds1307, we'll assume it'll send them all even though it doesn't have to
 second = Wire.receive(); 
 minute = Wire.receive(); 
 hour = Wire.receive(); 
 day_of_week=Wire.receive(); 
 day = Wire.receive(); 
 month = Wire.receive(); 
 year = Wire.receive(); 

 // Convert all the BCD values that might have "tens" to decimal.
 second = second/16 * 10 + second % 16;
 minute = minute/16 * 10 + minute % 16;
 hour = hour/16 * 10 + hour % 16;
 day = day/16 * 10 + day % 16;
 month = month/16 * 10 + month % 16;
 year = 2000 + year/16 * 10 + year % 16;
 
// Change times below to desired photo times. Copy & paste to add more photos per day.
// NOTE: for some reason, 8's & 9's cause an error, so don't use them on their own below;
// 18 & 19 work fine, but 08 & 09 do not.

 if (hour == 6) { if (minute == 00) { if (second == 00) { TakePhoto();}}}
 if (hour == 12) { if (minute == 00) { if (second == 00) { TakePhoto();}}}
 if (hour == 14) { if (minute == 00) { if (second == 00) { TakePhoto();}}}
 if (hour == 18) { if (minute == 00) { if (second == 00) { TakePhoto();}}}

 Serial.print(hour);
 print_prompt_P(PSTR(":"));
 if (minute < 10) { print_prompt_P(PSTR("0")); }
 Serial.print(minute);
 print_prompt_P(PSTR(":"));
 if (second < 10) { print_prompt_P(PSTR("0")); }
 Serial.print(second);
 print_prompt_P(PSTR(" on "));
 Serial.print(dow[day_of_week]);
 print_prompt_P(PSTR(", "));
 Serial.print(day);
 print_prompt_P(PSTR("/"));
 Serial.print(month);
 print_prompt_P(PSTR("/"));
 Serial.print(year);
 Serial.println();
  byte span = 20;  int aRead = 0;
  for (byte i = 0; i < span; i++) {        //loop to get average of 20 readings
    aRead = aRead + analogRead(ThermistorPIN);
    }
  aRead = aRead / span;
  temperature = Thermistor(aRead);

...continued next post...