SdFat support for SD.h style programs.

Other Hardware Storage
1

I have just posted a beta version of SdFat that supports the SD.h API on github GitHub - greiman/SdFat-beta: Beta SdFat for test of new features.

You now should be able to use SdFat with programs written for SD.h by replacing the #include <SD.h> with these two lines:

#include <SdFat.h>
SdFat SD;

The new File class in SdFat supports all the SD.h File member functions plus the SdFile member functions.

SdFat also supports the SD.h style open for the File class.

  File file;
  file = SD.open("test.txt", FILE_WRITE);
  if (!file) {
    Serial.println("open error");
  }

SdFat produces smaller faster programs than SD.h.

The SD.h ReadWrite.ino example in Arduino 1.06 uses 13,676 bytes of flash on an Uno.

With SdFat the ReadWrite.ino example uses 11,700 bytes of flash. Only the above change for the SD.h include is required.

The difference is smaller with the SD.h Datalogger.ino example. 14,756 for SD.h and 13,234 for SdFat.

For the SD.h listfiles.ino example, 14,696 for SD.h and 12,442 for SdFat.

I would appreciate any feedback on tests with other SD.h programs. I tested all the SD.h examples but suspect there will be problems with other programs.

I also suspect any problems will be easy to fix since SD.h is just a wrapper for a really old version of SdFat.

2

I did some tests with my Mega 2560 board. Using Arduino 1.5.8 BETA with -flto option.

The function FreeRam() is not included ?
http://github.com/arduino/Arduino/blob/master/libraries/SD/utility/SdFatUtil.h

I use a File.write(char *), and the parameter is either a string "Hello" or a variable that is a char , it causes : "error: invalid conversion from 'char' to 'uint8_t {aka unsigned char}' [-fpermissive]"

For now, I changed it into File.print().

The File.Write should allow text strings : http://arduino.cc/en/Reference/FileWrite

Retrieving webpage (already optimized for reading into a buffer of 128 bytes):
SD : 8.85 seconds.
SdFat : 6.08 seconds.

Reading and Writing 2GB Sd card is no problem.

Code size and sram usage:
SD : 43.888 bytes program. Global variables 2.311 bytes.
SdFat : 41.948 bytes program. Global variables 2.247 bytes.

No other problems, no memory leaks, no error messages.

3

Peter_n,

Thanks for trying the new File object features. I have lot of tests for other parts of SdFat but none for SD.h style programs so this is very helpful.

4

I read you post about formatting the SD card : Don't Format SD cards with OS utilities! - Storage - Arduino Forum
After using the SD Formatter 4.0 : 5.70 seconds for webpage :stuck_out_tongue:
https://www.sdcard.org/downloads/formatter_4/

I was reading the data one byte at a time with File.read() into a buffer of 128 bytes, that buffer is sent to the Ethernet.
After changing that by using File.readBytes() into the same buffer, it is slower, about 5.85 seconds.
Did you not implement it ? What did the linker use ? I might have done something wrong.

So I used your "File.read(buffer, nLength);", and read 128 bytes at a time. That was better : 4.04 seconds !

The default SD library function File.read() can only read one byte.

Even the Stream class is for just one byte.
http://arduino.cc/de/Reference/StreamRead
So I guess, the extension to File.read() will not be implemented, and the readBytes() will be used from now on.

Using File.readBytes() with the default Arduino SD library is not possible yet, I get many errors. So I don't know if that is faster with the default SD library.
"relocation truncated to fit: R_AVR_13_PCREL against symbol `__fp_round'"

5

"relocation truncated to fit: R_AVR_13_PCREL against symbol `__fp_round'"

This is bug in the Standard Arduino IDE. Do you have a large Mega 2560 program? I think it is fixed in the 1.5.7 or newer beta sketches fail to link with R_AVR_13_PCREL due to interrupt service routine too far for startup rjmp in vector [imported] · Issue #1071 · arduino/Arduino · GitHub

Using File.readBytes() with the default Arduino SD library is not possible yet, I get many errors. So I don't know if that is faster with the default SD library.

The standard Arduino Stream implements Stream::readBytes() using single byte reads so it will be slow.

size_t Stream::readBytes(char *buffer, size_t length)
{
  size_t count = 0;
  while (count < length) {
    int c = timedRead();
    if (c < 0) break;
    *buffer++ = (char)c;
    count++;
  }
  return count;
}

So I guess, the extension to File.read() will not be implemented, and the readBytes() will be used from now on.

The Arduino SD.h does have read(buf, length) but it is not documented.

  int File::read(void *buf, uint16_t nbyte);

Here is a test program that reads 128 byte records:

#include <SD.h>

File file;

void setup() {
  char buf[128];
  Serial.begin(9600);

  if (!SD.begin()) {
    Serial.println("begin failed");
    return;
  }
  file = SD.open("TEST.TXT");
  uint32_t t = millis();
  while (file.read(buf, sizeof(buf)) == sizeof(buf)) {
    // process data here
  }
  t = millis() - t;
  Serial.print("bytes read: ");
  Serial.println(file.position());
  Serial.print("time ms: ");
  Serial.println(t);

  file.close();
}

void loop() {}

I ran tests with a 1000 record file.

With SD.h the output is:

bytes read: 128000
time ms: 523

If I change File::read(buf, sizeof(buf) to File::readBytes(buf, sizeof(buf)) with SD.h, the output is:

bytes read: 128000
time ms: 6029

If I use SdFat with the new File::read(buf, sizeof(buf) the result is:

bytes read: 128000
time ms: 352

So SdFat with read(buf, length) is about 17 times faster than SD.h with readBytes(buf, length).

6

Hi I was thinking about moving to SdFat from SD in order to reverse traverse a directory, so I thought I should document the changes I made. I used the beta SdFat library from github https://github.com/greiman/SdFat-beta.git from. My own source code is in github, where I am running a due to give visual stimuli to fruit flies and record their response

  1. On replacing SD.h, I got a series of "volume" errors

WebServer.ino:150:1: error: 'SdVolume' does not name a type:
WebServer.ino:244:26: error: 'volume' was not declared in this scope

So I deleted the volume code completely -it seems unneeded in the SdFat code

2a) The next errors were related to the change from 'SdFile::readDir(dir_t&)' to 'FatFile::readDir(dir_t*)'
2b) and 'SdFile::open(SdFile&, const char*&, int)' to 'open(FatFileSystem* fs, const char* path, uint8_t oflag)'

These were easy to fix - now to plug in my due and see how it goes!
Thanks - keep up the good work