SD/MMC From the ground up

A number of people have asked if the information in the thread "SD card read/write with Arduino" be summarised for newcomers.

Well here it is. After reading this you should be able to:

  • Add an SD/MMC card to your Arduino.
  • Read data from and write data to it using the uFat and DevicePrint libraries.
  • Understand a few of the issues related to the process.
    This is my own personal take on the subject and I can only offer descriptions of and advice on using the code which I developed, so apologies if this comes over like an advert at times - it's truly not meant to be. I hope that opinions and advice about other code available will be added in subsequent posts and that this thread can be a focal point for those wising to take advantage of the work that has been contributed by the Arduino community at large.

On with the show.

MMC cards are a microcontroller's best friend. They can speak a serial protocol called SPI which is natively supported in the microcontroller hardware. I love 'em. You only need 4 IO lines to transmit commands and data to the card and receive data back. These connections go by a number of names but this is the set which will service you best:

/CS  (not card select)
CLK  (clock)
MOSI (master out, slave in)
MISO (master in, slave out)

Master in all cases is the Arduino. Digital pins 10 through 13 are the dedicated SPI connections. If your application already uses these you'll have to move things around! All the SD/MMC enabled boards I've seen use these pins. The information here should be applicable to anyone's SD/MMC enabled hardware.

Here's the MMC pinout...

...and here's how it needs to be connected:

MMC   Arduino
 1       10
 2       11
 3      gnd
 4      3v3
 5       13
 6      gnd
 7       12

MMC Cards require between 2.7 and 3.6 volts to operate. Low power varieties are also available that work down to 1.8V, though these are special and more rare. You can power a card from the Arduino's 3v3 output if it's available on your particular Arduino variety. There will need to be some level conversion on the /CS, CLK and MOSI lines, as these output 5V. As the Arduino regards 2.4V and above as a logic high, no level conversion needs to be done on the MISO line. There are some boards out there which operate at 3.3V natively, and for these boards all lines may be connected directly.

We do any neccessary 5V level conversion in the simplest way possible: with a voltage divider. There is great information elsewhere, linked below, so I won't go into any more detail than this:

VDD --[R1]-- v --[R2]-- GND

The voltage v can be calculated by using the following formula:

v = (VDD*R2) / (R1+R2)

In most cases you should use R1 = 1K8 and R2 = 3K3. Tapping the voltage at the centre of the resistor/resistor connection yields ~ 3.24V. Actual values will vary a little depending on the accuracy of the resistors.

Some cards can cope with voltages out of the specified range, and some cannot. I have cards which have functioned happily as high as 4.3V, and one in particular which developed a high fever followed by sudden explosive death when faced with the same levels... Ahem. The less said about that the better.

The product standard dictates the signal levels that you should aim for in your circuit.

Input HIGH voltage: 
[list][*]min:  0.625 * VDD
[*]max:  VDD + 0.3[/list]Input LOW voltage: 
[list][*]min:  VSS - 0.3
[*]max:  0.25 * VDD

[/list]
Here is what I consider to be the gold standard schematic produced by agent_orange, to be found with other good stuff at the start of the parent thread.

There are 3 kinds of socket available. MMC, SD and floppy-drive edge connector :smiley:

  • MMC sockets have 7 connectors and map 1:1 to the diagram above.
  • SD sockets can have up to 12 or 13 pins, but only the main 7 need to be used. The remainder are for detection of card insertion and write protect status of compatible SD cards. agent_orange's schematic shows a socket of this type.
  • Floppy edge-connector sockets are best for MMC cards (SD cards are thicker with ridges protecting the contacts) but are undoubtedly the cheapest! Check out the ingenious hack at Up All Night Robotics, linked below.
    There are many excellent introductions to the SPI protocol and I don't think that a discussion of it here would benefit anyone except the most inquisitive. There are a couple of links at the end of the post which should satisfy these curious cats.

It might not even help to discuss the MMC's command protocol over and above the fact that data transfers happen in 512 byte blocks. This can be changed, but I don't think there's a compelling reason to do this, especially as this most usually matches the sector size of the card.

Once the hardware is in place you'll need some software to drive it. As I said previously I'm hoping that people will contribute their code and experiences to this thread - I have only ever used one library to drive the card transfers. This was something that I adapted from the wonderful code of the SD2IEC project. I can't talk about 'The Roland Library' or any other as I don't know them. They should all be pretty much interchangable though.

For persistent storage on a card feel free to invent your own system. Reading and writing to the raw sectors is usually only a function call away. This is the least-cost entry into the storage arena. It's simple and reliable and you are in full control. But you will pay the price when it comes to moving the data around. You'll require raw sector access to the card from your computer which is certainly possible but inconvenient.

If you want to be able to log data and have it transferrable to your computer, then you'll need to be able to use the file system to some extent or other. This is a whole order of magnitude more complex than even the low level card protocols. Cards may be formatted with a number of filing systems, though generally you'll see one of the FAT (tm, probably) family.

This format uses tables of linked lists of cluster offsets (the File Allocation Tables or FATs) which can be used in turn to locate sectors. Clusters are a collection of multiple sectors, and the sectors per cluster count can vary depending on the formatting.

  • FAT12 is the oldest format out there. I still see cards formatted like this though, even today. It uses 12 bits to describe cluster offsets in the tables. It's limited to cards of up to 32Mb.
  • FAT16 is the most common. So common it's usually referred to as just FAT. It uses 16 bit tables and can describe the layout of cards up to 2Gb.
  • FAT32 is the daddy. If you need a 2Tb card, you'll need FAT32. As you've probably guessed it uses 32bit values for cluster offsets.
    Libraries do exist for talking to a FAT formatted device in the language of files. These are excellent for the times when you need that level of access and ease of use. They do tend to weigh in quite heavily though. In order to use a full filesystem approach you'll more than likely need one of the new ATMega328s, with their upgraded RAM and flash quotas, in your board.

Another approach is to swallow some small restrictions and use the very minimum of file system in order to use raw sector access to access your data whilst still retaining the ability to move files between Arduino and computer. uFat is a library that I wrote to do just this.

Just what restrictions might be necessary to go the low FAT route?

  • The donor/target file must be contiguous. When you delete or truncate files on a computer the sectors allocated to them are returned to a pool to be re-used when you next create or expand a file. This can mean that a file occupies sectors scattered all over the card. This is known as fragmentation, and it's why we have drive defragmenters. If you write to a fragmented file using raw sector access you might write over other unrelated data on the filesystem. Or the data you wrote won't be visible when you read the file on your computer. How to get around this? Format the card, create the donor/target file on the computer containing enough data to satisfy your needs and then copy it over to the card.
  • Target files should reside in the card root directory. No subdirectories are allowed - the code to traverse subdirectories is huge compared to the rest of the library and its inclusion is largely unneccessary. The root directory can hold 512 files, more than enough for most datalogging applications I suspect.
  • No long filenames. Stick to 8.3 and all will be well. Just like directories, the LFN system is built (some may say hacked, I couldn't possibly comment, ahem) upon the existing directory structure using other directory entries to hold the additional information. Again, I don't think the additional memory usage is contributing to anything sufficiently valuable.
    If you recall MMC sector transfers happen in bursts of 512 bytes. This isn't much fun if the data you're writing is of differing length packets. You'd have to keep track of your buffer usage and flush it to card at the right time. Plus, you'll need some extra help to get string output into a file instead of raw binary.

On top of (to the side of?) uFat I've reworked some code by bobemoe and produced DevicePrint, a hardware-agnostic library which enables formatted string output to any sector-based device. This library uses the newly-abstracted Print class available in Arduino-0012 and above. You get the same useful interface that you're used to using with the Serial class.

These libraries should provide you with most of your required daily intake of vitamins and minerals. If in doubt call your medical practitioner.

I think that's as much as I can say on this matter, I look forward to the conversations ahead. Huge thanks go to everyone who's contributed to my understanding of this subject and those who have helped me develop this code by using it!

Charlie

Links:

Original thread: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1206874649

MMC Product Manual: www.sandisk.com/Assets/File/OEM/Manuals/ProdManRS-MMCv1.3.pdf

MMC/uFat2 and Deviceprint: Arduino Nut: Libraries

The Roland: www.roland-riegel.de - sd-reader: MMC/SD/SDHC card library

AdaFruit: Audio Shield for Arduino

SD2IEC: http://sd2iec.de/

SPI introduction: Serial Peripheral Interface - Wikipedia

MMC card information at Elm: How to Use MMC/SDC

Fat module at Elm: FatFs - Generic FAT Filesystem Module

MMC card information at retroleum: http://www.retroleum.co.uk/mmc_cards.html

Voltage divider theory: Voltage divider - Wikipedia

Floppy-connector socketry: http://uanr.com/sdfloppy/

Thanks for reading! :sunglasses:

fantastic summary sirmorris!
If you could add a small snippet of code, something like the "hello world" for MMC, it would be perfect :slight_smile:

Don't forget mini-SD and micro-SD, both of which can also speak the SPI MMC protocol.

Someone suggested on another forum that the adapters that typically come with mini/microsd cards make pretty good sockets for the smaller cards. You can solder wires (carefully) to the full-sized adapter...

You're right! I should have mentioned that I'm taking MMC to mean all varieties: maxi, midi, mini, micro, nano, femto, and bob.

If you're feeling pushed for time and/or cash then you could also solder directly to the card, wires or a pin header - its connectors have standard .1" pitch. I feel dirty for just saying that.

Hello World coming up. I'll try to abstract it to show the required software components as opposed to the use of a library.

Nice work and summary.
Is it going to be included into the playground.

Could this work with the card socket on the official arduino ethernet card?
Or is the way its hooked up wrong for this method?

Thanks

Gordon

It works without a problem on lady ada's wave and GPS shields. I haven't experienced the official ethernet shield but I'd be very surprised if it used different pins, as the connections are defined by the Arduino's hardware SPI pin-out which is fixed.

I will put this up in the playground, I think.

Excellent summary sirmorris!
Thats more than worth to be placed in the playground!

Thanks! I looked today and I think I'll have to spend some time working out how to edit the wiki..!

Great work, nice write-up.

It took longer to find an SD card than it did to get it working!

Thanks.

This pic might be helpful for some:

It can be a little bit confusing, since the pins are not exactly ordered from 1-9.

Very cool Morris you've been a big help to the community. Thanks for all the help you've provided.

I did some quick testing using the sample DevicePrint sketch, and the fastest I can get any of my cards to write is about 500ms for 512bytes.

What is the bottleneck? How fast do you have the clock set? 4mhz? As I understand it, it has to be set at 4mhz during initialization, but can be bumped up to 40mhz(?) afterwards.

I took a look at the code. You have SPCR = B01010011, and in the comments I see "clock = f/4". Maybe I misunderstand, but isn't that clock/128? Shouldn't SPR1 and SPR0 be set to 0 to get clock/4?

I changed both to 0, and enabled SPI2X in SPSR. On my best card, the writes are now 80ms per 512byte block!

Is this safe? Is there any error checking?

You're right, my bad! I'll amend the code asap.

You should be able to drive the card as fast as Arduino can go without any problems.

This is a great library-- and it worked on my micro-SD card (Transcend 1G) right out of the chute.

One little problem I ran into is that if I have the SD card and an SCP1000 pressure sensor on the same SPI bus, I can't get them both to work.

I can get them to work individually in the same hardware configuration, and I can get the pressure sensor working OR the SD card working. (just by commenting out parts of the code) But if I initalize the SD card, the pressure sensor just returns 0x00's.

I know there's some funky SPI code to interface to a SD card-- is the card perhaps responding even if the CS line is high?

Also, on the library, mmc.h should have the #defines for the pins, not mmc.c -- since CS doesn't have to be Arduino 10 if you have multiple devices =)

Thanks again!

Excellent!

As for the multiple SPI slave situation - indeed, this is something that I've thought about but, like most things, I don't do anything about until it bites me :smiley:

So every device in the SPI chain needs its own /CS line. The only time you need to do anything funky with the one allocated to the MMC card is during initialisation. In order to 'wake it up' and put it in SPI mode you need to send it at least 72 clocks with CS high. I don't think that would cause too much of a problem in a multiple device chain as long as your other devices also have their select lines de-asserted.

If however the MMC will respond even with /CS high.. Oh oh. I've never investigated this. If this is the case then it's time for SoftSPI! This is something I've dabbled with when I was debugging something or other. It's a software implementation of SPI mode 0. You can allocate whichewver pins you like so there's no need to share any lines. I hope it doesn't come to this though... It's comparatively slow.

In practice it doesn't matter too much where the CS pin is defined if you use the MMC code as a library. It would have to be recompiled for every sketch that used it. A way around this would be to allow the CS pin be specified in a variable. It would need this facility in the libraries for each device though... Or not use a true library, instead copying the code to each sketch folder that required it.

C

Very good point! I can just re#define in my program and go to town.

I'd be interested to know if the card will respond with CS high-- the 72 clocks shouldn't do anything to the SCP1000, since it's CS is also high at the time. shrug might have something to do with my particular implementation.

Update: interestingly, it's the code and not the SD card-- I removed the card, but still had the software running. I had to completely remove power and replace power to get the SCP1000 to work with the new code. Bizzare...

Update: I downloaded the latest files and it works now ...

When I try to compile I get an error ... The last time I programmed in C was about 10 years ago, so forgive me if I should know this. I did try and figure it out myself, just not smart enough.

o: In function main': undefined reference to microfat2::walkDirectory(bool ()(directory_entry_t, unsigned int, void*), void*)


#include <WProgram.h>
#include <avr/pgmspace.h>
#include <microfat2.h>
#include <mmc.h>

byte sector_buffer[512];

char sprint_buffer[40];

// BEWARE - don't print strings longer than 39 characters!
// If you can't help it, adjust buffer size above.
//
void pprint(const char* s)
{
strcpy_P(sprint_buffer, (PGM_P)s);
Serial.print(sprint_buffer);
}

void error(const char* s)
{
pprint(PSTR("Error: "));
pprint(s);
pprint(PSTR(""));
for( /* ever */ ; ; )
{
digitalWrite(13, (millis() / 250) & 1);
}
}

bool showDirectory_walkerfn(directory_entry_t* directory_entry_data, unsigned index, void* user_data)
{
int* count = (int*)user_data;

Serial.print(index, DEC);
Serial.print(' ');

// Terminate the filename string.
// This is deliberately corrupting the buffer data, but that's ok.
directory_entry_data->filespec[11] = 0;

Serial.println(directory_entry_data->filespec);

// Increase 'seen file' count
*count = (*count)+1;

// don't stop
return false;
}

void showDirectory(void)
{
int count = 0;

pprint(PSTR("Directory of files on card:\n\n"));

microfat2::walkDirectory(showDirectory_walkerfn, &count);

pprint(PSTR("\n"));
Serial.print(count, DEC);
pprint(PSTR(" files found.\n\n"));
}

void setup(void)
{
Serial.begin(115200);

pprint(PSTR("uFat2 demonstration\n"));

if (mmc::initialize() != RES_OK)
{
error(PSTR("mmc init failed.\n"));
}

// Pass in the sector-sized buffer we'll be using ourselves later.
// uFat doesn't own it, it just needs to use it temporarily.
// We also pass in the address of a function that is used to read
// sectors from our device.
//
if (!microfat2::initialize(sector_buffer, &mmc::readSectors))
{
error(PSTR("uFat init failed.\n"));
}

showDirectory();

unsigned long sector;
unsigned long byteSize;

if (microfat2::locateFileStart(PSTR("DATA BIN"), sector, byteSize))
{
if (byteSize >= 512)
{
if (RES_OK == mmc::readSectors(sector_buffer, sector, 1))
{
for (int i = 0; i < 512; ++i)
{
sector_buffer = sector_buffer + 1;
* }*
* if (RES_OK == mmc::writeSectors(sector_buffer, sector, 1))
_
{_
_
pprint(PSTR("Written to data.bin OK!"));_
_
}_
_
else*_
* {*
* pprint(PSTR("Failed to write updated data."));*
* }*
* }*
* else*
* {*
* pprint(PSTR("Failed to read data.bin."));*
* }*
* }*
* else*
* {*
* error(PSTR("Found data.bin, but it's too small."));*
* }*
* }*
* else*
* {*
* pprint(PSTR("data.bin not present on card."));*
* }*
}
void loop(void)
{
* digitalWrite(13, (millis() / 1000) & 1);*
}