Cant get SD card to flush / sync :(

Hey Guys

New to Arduino and am struggling to get logging to SD card working. I've tried everything - really need some help.

I'm using a freetronics etherten and trying to use the standard SD library to print log messages to a txt file in the root folder of an SD card. If I open the file, print the message and close the file, it works fine - however I'm thinking that's not very efficient to close and open the file all the time, so what I wanted to do was print log messages and then only periodically flush them to the card. The issue is flush isn't working for me (when I reset the program no data is written to the card even after calling flush - the file has been created but it's just empty). The flush implementation in SD library returns void (as it needs to comply with the stream implementation), however i've found the underlying operation is failing (put print statements in the code to trace back my issue:

SdFile::sync is returning false, due to this:
dir_t* d = cacheDirEntry(SdVolume::CACHE_FOR_WRITE);
if (!d) return false;

And so I've traced it through to find the issue is in Sd2Card::writeData
Basically this section:

status_ = spiRec();
if ((status_ & DATA_RES_MASK) != DATA_RES_ACCEPTED) ....

status_ for me is 0 so the write data function returns false.

I'm well over my head here - I have no idea what's wrong. I've tried with Arduino 1.0.2 and 1.0.4. I've tried formatted my SD card (1GB Sandisk) with the proper SDFormatter and with windows. I can see that SD is not supplying anywhere near the latest version of sdfat I've tried modifying it to use a later one, but didn't get anywhere with that either. I don't understand what's going wrong - I cant seem to turn up similar problems in google...

Here's the basis of my code:

setup:
	pinMode(SS_SD_OUT, OUTPUT);
	pinMode(SS_ETHERNET_OUT, OUTPUT);
	digitalWrite(SS_ETHERNET_OUT, HIGH);	// Supposedly disables the ethernet
	if( SD.begin(SS_SD_OUT) )
	{
		sdLogFile = SD.open("datalog.txt",  O_CREAT | O_WRITE | O_APPEND);
	}

	if( sdLogFile )
	{
		displayMessage("SD card initialized", mlINFO);
	}

loop:
        print stuff to sdLogFile
        call flush every 3 seconds
        delay(10);

Hi,
just as a wild guess: most problems I had with my SD Card tests where caused by memory overflows (up to that Arduino reset automatically or even some fuses blown such that I had to reflash the bootloader).
I collected some of my learnings here: Experiences with Arduino,SD Card and Memory consumption – Now It works !!! – robertgetzner.com

Maybe you could also try something else

  1. somewhere explicitly close the file (at least for a test), this is not shown in the coding parts you posted
  2. test without anything else in the sketch (e.g. don´t use diplaymessage but only blink an LED)
  3. Try another library e.g. Fat16 Lib

HTH, Robert

Thanks Rob - looks like a great resource there on your blog I'll have a read through it over the long weekend.

Re your suggestions there:

  1. it's buried in my opening post - but closing the file works. What I dont understand though - almost all the examples on the net show opening and closing the file within loop - i would have thought that's very inefficient compared to holding the file open and just flushing periodically but maybe I'm wrong about that - i think that's part of the reason I cant find much googling on this - most people aren't using flush.
  2. i'll try that to rule out memory issues - it'll also help as i'll be able to post my full code. I did run into a memory issue when i added too many debug print statements so I'm obviously hovering fairly close, however the lack of flush has been very consistent despite me trying lots of different things, so I'm not convinced it's memory at this point
  3. I'll try Fat16 - i was hoping to use the supplied SD wrapper library - thinking it should work since it comes with Arduino base install and it's nice and simple - it's also hard to get small SD cards these days (i need to stay under 2gb don't i for fat16?)

Thanks
Mick

hi,

re 1) sorry, missed the close in your initial post
for your question within 1) : why not close () open() the file e.g. every 10 seconds in the loop ? I don´t know your use case, you are doing this for. Having an explicit close () somewhere maybe also helps to 'secure' the data on the SD in case your arduino looses power or worse. mhhh but I dunno if the file will be corrept nonetheless, when close() wasn´t the last operation

re 3) Fat16 has some limitations if I recall correctly (8.3 filenames and root folder only), but for most cases this will be sufficient - unless you try to build an OS, image gallery or blog on Arduino :). I use a 2GB microSD and for me this works fine.
But I didn´t come across a library with smaller memory footprint. And the 1K on the 328p is not much...

Hi - thanks for your help. I'm happy to report I've got it working!

I took your suggestion to cut right back my code and slowly added things until it stopped working - turns out the problem was a conflict with the motor shield I'm using - it was making use of timer 1 and enabling both pins 9 and 10 for PWM output. Although the SD card is CS pin 4, pin 10 is the CS pin for the ethernet in my board and the PWM output was messing with the SPI somehow. It's a bit frustrating they would choose those particular pins as timer 1 is the only 16 bit timer and to have it clash with the ethernet is pretty frustrating. Anyway fortunately I'm only using 1 motor on the dual motor shield, so I was able to steal the bit of code from the Timer1 library to disable the PWM output for pin 10 (I only need pin 9 for the motor speed control). All is working now - I can leave the file open and just periodically flush the data to secure it on the SD card.

I don't open and close files all the time, I open them and write to them, the problem I run into
is that about a third of the time, when I stop the arduino, the resulting file is unusable.

What I ended up doing, is adding a button which causes the file to be closed. So when I am
about to unplug the arduino, I push the button, file is closed properly. So far this always
works and the file is readable when I transfer the SD card to the PC.

gr8 to hear that it worked & many thanks for reporting back! By this we can use it as a good example how this procedure of finding conflicts by different libraries can help :slight_smile:

Could you try to keep an eye on the reliability of the flush() without close(). I believe it could be a general pattern to only use flush periodically, but for that some testing would be good :wink:

Which SD and motor shield libraries/versions did you use ? If other people in future could find this thread, when searching for this combination of libraries, this will be a big help for them.

Thx, Robert