Compiling an SD/MMC and fat16 library

@agent_orange:

that code really helped! thanks a lot! Works perfect!

At the moment i am still over it, run i a little question ... What about the 3.3V! I believe i read some time ago it´s only powered when connected to usb, got it from ftdi chip? Is that right? So i would need for sure an additional voltage regulator for external power supply.
Does someone know exactly?

Now i have arduino connected to usb with a kingston 1G card in the slot. Powered from diecimilia 3.3V supply and added a 0.47uF Elco to 3.3V power supply of the card. For the moment no problems with it.

First i will power the board with 9V alkaline for testing, later a 3.7VLiPo with stepUp. (does someone have a circuit for that? Will also add an Max1811 for LiPo charging when on usb. Have the max appnote for that.
But if someone has a working version for arduino would be fine.)

Nice sun at the moment here, hopefully get it working for a walk in the outside soon!

i will think before i post.. ;D

got the simple answer about 3.3V 2min later with measurement equipment: 3.3V pin works with a 9V alkaline.

How can i get an unsigned long int converted to it´s bytes?
I want to store the offset in the start range of the card, the first four bytes.
So this could be read the next time it starts recording and does not overwrite the record from before.

I thought code like this could help, but i googled extensively for a solution, but didn´t found something really helpful.
Here´s how i thought:

unsigned long int sdoffset=1024;  // r/w offset
// offset 0 to 1023 is for rw offset and whatever storage.
// for the first usage have to write zeroes to that.

void checkStartOffset(){
  byte offsetstored[4];
  sd_raw_read(0, offsetstored ,4);
  sdoffset=(unsigned long int)offsetstored;
  Serial.print("sdoffset read: ");
  Serial.println(sdoffset);
}
void WriteLastStartOffset(){
  char offsetstored[4];
  offsetstored=(byte)sdoffset;
  sd_raw_write(0, offsetstored ,4);
}

the start check compiles, but don´t know at the moment if that works. The read function will not compile.
Could someone point me into the right direction?

Thank you very much for any help with this!

Ok I'll try and expalin but someone else probably can do it better. If you are writing numbers larger than bytes (8bits = 2^8-1 = 255) then you need to split the number up into separate bytes. I'll give a code example. I wanted to store the values of the arduino ADC in the memory card. Values from the ADC are 10bit (10bits = 2^10-1 = 1023). So the 10bit number needs to be split into two values to be able to be stored in 8bits.

// Store my 10bit value in an int (NOT a byte, which is only 8bits) so 16 bit number. High(1111 1111) Low (1111 1111)
int temp;

byte low;    // Variable to store the low byte in
byte high;   // Variable to store the high byte in

temp=analogRead(0); //Read in the value

// AND the number with 255 which wipes the 8bits at front of number 
// The front 8 bits gets wiped->High(1111 1111) Low(1111 1111) = High(0000 0000) Low(1111 1111)
// So you can store half the number in the low byte variable
low=temp&0xFF;     


// Rotate the number 8 bits right to get the high bits.
high=temp>>8;

So you can then store each byte in an array and then write it to disc. To read back from the disc you read in each byte and then join them together again.

byte low;
byte high;
byte info[2];
int value;

sd_raw_read(0,info,2); // Read the 2 values from disc
low=info[0];  // store low byte
high=info[1]; //Store high byte
result=high<<8; // rotate left 8 bits the high value and store in result
Serial.print(result+low,DEC); //Add low and high together to get the original number you split up into 2 bytes

The easiest way to see how this all works is to try this code and use Serial.print after each operation and print the numbers in binary. Then you can see how the bit manipulations are working. For larger values long int etc, you have to split the number up into 8bytes and then store each byte individually. So if you wanted to store a 32 bit number you would need to split it into 4 bytes and then write each byte to disc.

Well, hopefully I explained it ok :slight_smile:

@agent_orange
Thanks again a lot! I think i´ve got it mostly. I made for test some handy defines:

#define HIGHBYTELONG(x)  ((byte)((x) >> 24))
#define LOWBYTELONG(x)  ((byte)((x)  >> 16))
#define HIGHBYTE(x)  ((byte)((x) >> 8))
#define LOWBYTE(x)  ((byte)((x) & 0x00ff))
#define WORD(x,y)   (( ( (byte)(x) ) << 8) | ((byte)(y)))

unsigned long int ulnumber = 0xABCDEF89;  


void setup() 
{ 
  Serial.begin(9600); 
  Serial.println(ulnumber, HEX);   // prints value as string in binary (base 2) 
  byte hbl = HIGHBYTELONG(ulnumber);
  Serial.println(hbl, HEX);   
  byte lbl = LOWBYTELONG(ulnumber);
  Serial.println(lbl, HEX);   
  byte hb = HIGHBYTE(ulnumber);
  Serial.println(hb, HEX);   
  byte lb = LOWBYTE(ulnumber);
  Serial.println(lb, HEX);   
  unsigned int wi = WORD(hbl,lbl);
  Serial.println(wi, HEX);   
  unsigned int wi2 = WORD(hb,lb);
  Serial.println(wi2, HEX);   
  unsigned long int wix = (( (((unsigned long int) wi) <<16)) | ((unsigned  int) wi2));
  Serial.println(wix, HEX);   
} 


void loop() 
{ 
}

i get the following output:

ABCDEF89
AB
CD
EF
89
ABCD
EF89
ABCDEF89

i took hex numbers, so you can see it directly if it works correct.
But i could not make a working define for the unsigned long int directly. But for now i have what i need to go on.
Thanks a lot again! :slight_smile:

Made yesterday in the late evening a little walk with the raw boxed logger device. It worked so far. But i must have had some worse serial connector or the cold was to much, @-4Celsius. Many track point missed caused by worse checksums, also the serial connection had too much failures or the mmc? . I´ll make today a walk again.
The sd_raw write works really fine now, but it is much more slower than the gps. I expected it been faster than the gps. So i changed for test the buffer to 512bytes, thats the bulk it always wants to write i think.
i´ll post that code later.

--- more info to come
added a missing "#" in code section

For an outside was no time, but tested inside. Now the data looks quite good. Saving the offset seams to work now. So the hole thing is still quite dirty, but it seems to work so far.
I take a 1G Kingston card. Data write section starts at offset 4096. The first 4 bytes on the card are for saving the actual offset.
At power on it checks the offset on the card, so it should be cleared on a fresh card or when you want to write from beginning. If it finds offset data, writing starts from there.
(On the PC i read it with HxD http://www.mh-nexus.de thats a raw reader i found for free.
You can read the data with it and clear the card if you want. Useful for test anyway.)
Then the arduino fills the write buffer with 128 bytes, the length of the serial buffer, i thought that may increase speed. Tried that with 512 bytes, i read in the sd card data sheet that this is always that bulk it writes. But that didn´t wanted to work. Went back to 128, no problem. After fill it writes the data on the card, writes the new offset and flushes the cache. Without flushing i had sometimes no data on the card.

After an 1hour test run i converted the data with GPS Babel http://www.gpsbabel.org/ to kml and checked with g**gle earth. For the test run i was on USB.
For the 3.3V i added a low drop regulator a LF33CV. I simply connected it to 5V and ground on my shield, the out pin directly to the Vin of the card. Before i had added an elco to 3.3 and ground at the card, so there was no need for an additional one. The first proposed condensator i saw no need for, cause of the stabilized 5V, but that may not be correct, but works. With my Velleman personal scope voltage looks fine without ripples, ups and downs. The Velleman may not be the best, but it´s very useful for me. Mostly better than a multimeter.

Ok, the code: Binary sketch size: 5348 bytes (of a 14336 byte maximum)
in the next post...

... continues from previous post

/* work in progress ... 2008 02 17
 * it´s still quite dirty! take care!
 *
 * using stuff from:
 * arduino.cc -> got the basics, they made it easy and fun!
 * Roland Riegel http://www.roland-riegel.de/sd-reader/ -> the awesome mmc&fat libs
 * Ulrich Radig http://www.ulrichradig.de/home/index.php/avr/mmc-sd -> hardware hint
 *                 has also made an interesting mmc & fat lib, but not used here.
 * www.mikrocontroller.net -> got some good soft-&hardware hints (german language)
 * http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1199737337 -> the important things for the code
 *                 from the nice people here to get started with this project.
 *
 * www.navilock.de -> the gps module (got the rs232, better would be the ttl level one)
 * so i had to add a max232 IC for serial communication. with ttl no need for that.
 *    http://www.navilock.de/produkte/gruppen/13/Boards_und_Module/60409_NL-500ERS_Sirf3_RS232_Modul.html
  *
 * This little project is for getting the basics of gps reading 
 * and long time data storage together.
 * Should get a useful gps-data-logger when ready.
 *
 * Ideas&ToDo:
 * buttons for r/w/a (read write append) functionality?
 * Adding a Siemens S65 LCD display.
 * Connect it to a WiiRemote -- log acceleration, degree, communicate with BT &
 *    add XBeePro for reality mesh network activity (great potential in it!)
 *    maybe for exploring land- and city scapes, reality gaming... with a group of people.
 */
// including the mmc libraries for raw r/w support:
#include <sd-reader_config.h>
#include <sd_raw.h>
#include <sd_raw_config.h>

// defines for splitting the unsigned long int into bytes for storage:
#define HIGHBYTELONG(x)  ((byte)((x) >> 24))
#define LOWBYTELONG(x)  ((byte)((x)  >> 16))
#define HIGHBYTE(x)  ((byte)((x) >> 8))
#define LOWBYTE(x)  ((byte)((x) & 0x00ff))
#define WORD(x,y)   (( ( (byte)(x) ) << 8) | ((byte)(y)))

unsigned long int sdoffset=4096;  // r/w offset
// offset 0 to 4096 is for rw offset and whatever future storage.
// for the first usage have to write zeroes to the card.
// the last offset on my 1G card is 1009254384
// get as much buffer length as the serial port has:
int in_buf_length=128;
byte incomingBuffer[128];  // for incoming serial data

// optical function control:
int ledWrite = 9; 
int ledRead = 8; 

void setup() {
  pinMode(ledWrite, OUTPUT);      // sets the digital pin as output
  pinMode(ledRead, OUTPUT);      // sets the digital pin as output

  //Init of the MMC/SD-Card
  // give it some time
  delay(500);
  digitalWrite(ledRead, HIGH);   // sets the Read-LED on
  digitalWrite(ledWrite, HIGH);   // sets the Write-LED on
  sd_raw_init ();
  delay(500);
  digitalWrite(ledRead, LOW);   // sets the Reaad-LED off
  digitalWrite(ledWrite, LOW);   // sets the Write-LED off
  delay(500);
  // and a second time...
  sd_raw_init ();
  // let´s see where we go:
  for (int i=0; i <= 3; i++){
    digitalWrite(ledRead, HIGH);   // sets the Read-LED on
    digitalWrite(ledWrite, HIGH);   // sets the Read-LED on
    delay(300);
    digitalWrite(ledRead, LOW);   // sets the Read-LED off
    digitalWrite(ledWrite, LOW);   // sets the Read-LED off
  } 
  Serial.begin(9600);      // navilock works with 9600bps from scratch
  // For programming you have to disconnect the gps physically! 
  // check where we stopped writing the last time:
  checkStartOffset();
}


void loop() {
  digitalWrite(ledRead, HIGH);   // sets the Read-LED on
  fillSerialbuffer();
  digitalWrite(ledRead, LOW);   // sets the Read-LED off
  digitalWrite(ledWrite, HIGH);   // sets the Write-LED on
  sd_raw_write( sdoffset , incomingBuffer , in_buf_length );
  sdoffset=sdoffset+in_buf_length;
  writeLastStartOffset();
  sd_raw_sync ();      
  digitalWrite(ledWrite, LOW);   // sets the Write-LED off
}


void writeLastStartOffset(){
  byte offset[4];
  offset[0]=(HIGHBYTELONG(sdoffset));
  offset[1]=(LOWBYTELONG(sdoffset));
  offset[2]=(HIGHBYTE(sdoffset));
  offset[3]=(LOWBYTE(sdoffset));

  sd_raw_write(0, offset ,4);
}
void fillSerialbuffer(){
  // we have enough storage with 1G
  // so we take the hole input from gps.
  // don´t care about single lines, get the whole stuff!
  // read one serial buffer with length of 128:
  int var = 0;
  while(var < in_buf_length-1){
    if (Serial.available() > 0) {
      incomingBuffer[var] = Serial.read();
      var++;
    }
  }
}
void checkStartOffset(){
  byte offsetstored[4];
  sd_raw_read(0, offsetstored ,4);
  unsigned int wri = WORD(offsetstored[0],offsetstored[1]);
  unsigned int wri2 = WORD(offsetstored[2],offsetstored[3]);
  unsigned long int readdoffset=( (((unsigned long int) wri) <<16)) | ((unsigned  int) wri2);
  // when it is a clean card with 00 00 00 00 on it,
  // start anyways at the right offset.
  if (readdoffset < 4096)
  {
    sdoffset=4096;
  }
  else
  {
    sdoffset=readdoffset;
  }
}

i had to cut out the out-commented debug code, it was to long.

@agent_orange
Thanks again for help! I feel having hijacked this thread, hope that is ok for everyone? Should i stay here with future additions and improvements? Because it is not purely the topic of the post now?

If there is serious interest i can describe the project with more detail, pictures and so on.

Tomorrow i will try to log my way to work. I´ll post information especially when success is not satisfying.

Heyho!

So i had a walk in the outside about 2 hours. The old battery finished after 15mins, i added a fresh 9V block and the data was appended as it should. There were much less errors in the checksums as the first time. I think the 3.3V regulator was the right decision to add to the circuit. The leds blink much more the same tact. To be on the safe side i would say add a low drop regulator, maybe another card takes more current so you would need it anyways. And it´s only the cost of 0.75?.

I also made some measurement:

I measured with an 1R in series with the arduino and got
150mA at work. (140mA-175mA) at logging after init sequence, depending on write frequency (has to be analysed why sometimes more to write)
180mA at init sequence

25mA Arduino alone
78mA " & MAX232
80mA " " & LED stuff
140mA " " " without card

so the nice thing is, that sd-card writing is not current hungry! Gps is.
But you should have a big buffer, would be interesting to see with a good measurent equipment how the card behaves. I think there may be really short big peaks of current. An sd-card has a minimum maxWrite frequency of 25MHz per datasheet. Too short for me to see.

But when i look at it with my scope when i put the card into the slot without gps, i see that the card has it´s own logic. ca. every second it makes pulses about 50mA, so it may try to communicate or whatever.

That may not be super correct measured, but the whole measurement should be proportional right.
Same 1R, same fresh 4R25 6V battery.

That´s it about the electrical stuff. Hope that helps anyone.

Good stuff, strohhalm. :slight_smile: I'm hoping to implement a similar datalogger solution for a GPS and OBD2 (in-car diagnostics and stats). It'll be a while before I can get my head around what you've done, but this looks like a great start.

I'm sorry...I'm ok with the software part but I cannot understand where did you connected P_B0 ... P_B3 on the Arduino board.
Can you please explain this part a little better?

Thank you, Nuno

STOP!
Nevermind...just found :-[

Hi Nuno

I was embarrassed with it like you when i first saw it. but next moment I thought it could be the port number of Atmega168 chip

As you know, the Port B0 ~ PortB4 are pin no. 14~17. try it :slight_smile:

Strohhalm~ please give us the real answer ~

P.S.

And I thank you all guys above for all of your efforts. It'll be very helpful to my project

Thank you very much~

Hiya.

If a standard FAT library is too heavyweight for your application and you don't fancy raw card reading & writing here's a halfway-house solution that I've written:

I'm very happy to help with enhancements or explanations.

Charlie

I'm still trying to make it works...that circuit just seems to do...nothing.
I checked every wire thousand times but it doesn't work. :frowning:

Working with MMC/SD cards can be frustrating. I had a hard time making my first effort work.. :frowning:

Are you using an MMC or an SD card? Have you tried different cards? The schematic you're using is good, you can cross-reference your work with a different version which shows the wiring for an SD socket here http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1206874649/1#0. The main difference is that this version powers the card from an Arduino's 3v3 supply whereas the version you based your circuit on is using diodes to drop the 5v to around 3v3. Other than that both card types can be addressed in the same way.

I've come in late to this conversation - what do you want to achieve? What code are you using?

Working with MMC/SD cards can be frustrating. I had a hard time making my first effort work.. :frowning:

Are you using an MMC or an SD card? Have you tried different cards? The schematic you're using is good, you can cross-reference your work with a different version which shows the wiring for an SD socket here http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1206874649/1#0. The main difference is that this version powers the card from an Arduino's 3v3 supply whereas the version you based your circuit on is using diodes to drop the 5v to around 3v3. Other than that both card types can be addressed in the same way.

I've come in late to this conversation - what do you want to achieve? What code are you using?

I'm already using the schematic without the two diodes, but I tried that one too.
I really don't know how to find the problem...if it was a SW I could use a debugger, but here?
Anyway...I'm using a Kingston 64mb SD. The card works fine in my PC even after all the soldering I've done on the card itself (I can not find a socket...) so the problem is not there.
I checked the value of the resistors and thery are all fine...
Connected VCC on SD to 3v3 on Arduino, GND on SD to GND beside 3v3,
pin 1 on SD to 10 on Arduino
pin 2 on SD to 11 on Arduino
pin 3 is GND
pin 4 is VCC
pin 5 on SD to 13 on Arduino
pin 6 is GND
pin 7 on SD to 12 on Arduino (without any resistor because it's from SD to Arduino so it's already 3.3v)

Am I forgetting something? :cry:

I'm trying to build a simple 3-channel datalogger...the input part is easy but useless if I have no place to store the data I collect...

Here's one of the best introductions to the software side of mmc interfacing:

http://www.retroleum.co.uk/mmc_cards.html

Can you point me to the software that you use to initialise and read the card? I'll have a look over it.

Here's one of the best introductions to the software side of mmc interfacing:

http://www.retroleum.co.uk/mmc_cards.html

Can you point me to the software that you use to initialise and read the card? I'll have a look over it.

I'm using the same libraries used by AgentOrange in the post that you linked me.
Anyway the init part is the following

uint8_t sd_raw_init()
{
    /* enable inputs for reading card status */
    configure_pin_available();
    configure_pin_locked();

    /* enable outputs for MOSI, SCK, SS, input for MISO */
    configure_pin_mosi();
    configure_pin_sck();
    configure_pin_ss();
    configure_pin_miso();

    unselect_card();

    /* initialize SPI with lowest frequency; max. 400kHz during identification mode of card */
    SPCR = (0 << SPIE) | /* SPI Interrupt Enable */
           (1 << SPE)  | /* SPI Enable */
           (0 << DORD) | /* Data Order: MSB first */
           (1 << MSTR) | /* Master mode */
           (0 << CPOL) | /* Clock Polarity: SCK low when idle */
           (0 << CPHA) | /* Clock Phase: sample on rising SCK edge */
           (1 << SPR1) | /* Clock Frequency: f_OSC / 128 */
           (1 << SPR0);
    SPSR &= ~(1 << SPI2X); /* No doubled clock frequency */

    /* initialization procedure */
    
    if(!sd_raw_available())
        return 0;

    /* card needs 74 cycles minimum to start up */
    for(uint8_t i = 0; i < 10; ++i)
    {
        /* wait 8 clock cycles */
        sd_raw_rec_byte();
    }

    /* address card */
    select_card();

    /* reset card */
    uint8_t response;
    for(uint16_t i = 0; ; ++i)
    {
        response = sd_raw_send_command_r1(CMD_GO_IDLE_STATE, 0);
        if(response == (1 << R1_IDLE_STATE))
            break;

        if(i == 0x1ff)
        {
            unselect_card();
            return 0;
        }
    }
    
    /* wait for card to get ready */
    for(uint16_t i = 0; ; ++i)
    {
        response = sd_raw_send_command_r1(CMD_SEND_OP_COND, 0);
        if(!(response & (1 << R1_IDLE_STATE)))
            break;

        if(i == 0x7fff)
        {
            unselect_card();
            return 0;
        }
    }

    /* set block size to 512 bytes */
    if(sd_raw_send_command_r1(CMD_SET_BLOCKLEN, 512))
    {
        unselect_card();
        return 0;
    }

    /* deaddress card */
    unselect_card();

    /* switch to highest SPI frequency possible */
    SPCR &= ~((1 << SPR1) | (1 << SPR0)); /* Clock Frequency: f_OSC / 4 */
    SPSR |= (1 << SPI2X); /* Doubled Clock Frequency: f_OSC / 2 */

#if !SD_RAW_SAVE_RAM
    /* the first block is likely to be accessed first, so precache it here */
    raw_block_address = 0xffffffff;
#if SD_RAW_WRITE_BUFFERING
    raw_block_written = 1;
#endif
    if(!sd_raw_read(0, raw_block, sizeof(raw_block)))
        return 0;
#endif

    return 1;
}

It seems to be ok, does everything explained in your link. I'll try to debug the library to see where it stops.
What do you think about the code?

Thank you for you help

The code looks good, I'll run it against my hardware and let you know what happens.

This is a long shot but could you email me a digi picture of your card with its soldered connections? I've PM'd you my email address.

The code looks good, I'll run it against my hardware and let you know what happens.

This is a long shot but could you email me a digi picture of your card with its soldered connections? I've PM'd you my email address.

Yes, I'll send you a pic as soon as I can...I'm quite busy rigth now with 2 exams in 2 days :smiley:
Check your email in 6-7 hours.

thank you

Post up the code your using in your arduino sketch., that could help. If the card does nothing, then try making a simple sketch which just initializes the card and then prints out the sd card info. That should tell you if the card was initialized and is working or not.

Also, be aware that once you write to the card from the arduino you cannot read it in the pc again until you reformat it. Because you are writing straight onto the card which overwrites the FAT table.

If the card can be initialized and the disk info printed out then you "should" be able to read/write to it(murphy usually stuffs things up though (^_^)).