Here is a sketch that finds the next file name in optimal time, one pass through the directory file.

This sketch will create ten empty files each time it is run.

Code:

#include <SD.h>

#define IMAGE_DIR "/DCIM/100ARDUI"

// SD chip select pin
const uint8_t SD_CS_PIN = SS;

//
// return next file index or negative for error
int nextNumber() {
  dir_t dir;
  File dirFile;
  int maxIndex = -1;
 
  // open directory file
  dirFile = SD.open(IMAGE_DIR, O_READ);

  if (!dirFile){
    // open of dir file failed
    return -1;
  }
  // read all entries in directory
  while (dirFile.available()) {
    if (dirFile.read(&dir, 32) != 32) {
      dirFile.close();
      // read of dir file failed
      return -2;
    }
    // break if all entries read
    if (dir.name[0] == 0) break;
   
    // check for IMGLnnnn.JPG
    if (strncmp((char*)dir.name, "IMGL", 4)
      || strncmp((char*)&dir.name[8], "JPG", 3)) {
      continue;
    }
    int n = 0;
    for (int i = 4; i < 8; i++) {
      int c = dir.name[i] - '0';
      if (c < 0 || c > 9) {
        n = -1;
        break;
      }
      n = 10*n + c;
    }
    if (n > maxIndex) maxIndex = n;
  }
  dirFile.close();
  return maxIndex + 1;
}


void setup() {
  char path[32];
  File image;

  Serial.begin(9600);
 
  Serial.println("type a character to run");
  while (!Serial.available());
  if(!SD.begin(SD_CS_PIN)) {
    Serial.println("begin");
    while(1);
  }
  for (int i = 0; i < 10; i++) {
    int n = nextNumber();
    if (n < 0) {
      Serial.print("ERROR ");
      Serial.println(n);
      while(1);
    }
    sprintf(path, "%s/IMGL%04d.JPG", IMAGE_DIR, n);
    Serial.println(path);
    image = SD.open(path, FILE_WRITE);
    if (!image) {
      Serial.println("open image");
      while(1);
    }
    image.close();
  }
}

void loop() {
}

Here is typical output:

type a character to run
/DCIM/100ARDUI/IMGL0021.JPG
/DCIM/100ARDUI/IMGL0022.JPG
/DCIM/100ARDUI/IMGL0023.JPG
/DCIM/100ARDUI/IMGL0024.JPG
/DCIM/100ARDUI/IMGL0025.JPG
/DCIM/100ARDUI/IMGL0026.JPG
/DCIM/100ARDUI/IMGL0027.JPG
/DCIM/100ARDUI/IMGL0028.JPG
/DCIM/100ARDUI/IMGL0029.JPG
/DCIM/100ARDUI/IMGL0030.JPG

Does any one know if / how it's possible to use a straight line of pins (i.e. pins 46,48,50,52)on the Arduino MEGA 2560

You can not use 46 and 48.  The Mega SPI pins are 50-53.

See http://arduino.cc/en/Main/ArduinoBoardMega2560

SPI: 50 (MISO), 51 (MOSI), 52 (SCK), 53 (SS). These pins support SPI communication using the SPI library. The SPI pins are also broken out on the ICSP header, which is physically compatible with the Uno, Duemilanove and Diecimila.

The Arduino SPI pins are:

SPI	Uno	Mega
SS	10	53
MOSI	11	51
MISO	12	50
SCK	13	52

So:
11>51
12>50
13>52

Any pin can be used for SD chip select.

Code:

  SD.begin(CS_PIN);

For more see http://arduino.cc/en/Reference/SPI.

If you read all 144 bytes in a column with single read, you can reduce the average time to 350-500 microseconds for a 144 byte read.

Here are times for a low cost microSD

Maximum latency: 3108 usec, Minimum Latency: 108 usec, Avg Latency: 440 usec

Here are results for an Industrial full size SD

Maximum latency: 2020 usec, Minimum Latency: 108 usec, Avg Latency: 385 usec

These results are from the SdFat bench.ino example with a 144 byte buffer size.

Code:

#define BUF_SIZE 1144

Here is a sketch that prints short and long file names for files in the volume working directory.

Code:

#include <SdFat.h>
SdFat sd;
const uint8_t SD_CS_PIN = SS;
//------------------------------------------------------------------------------
// does not implement directory byte 12 for lowercase bits with 8.3 names.
// bit 0X10 means lowercase extension and bit 0X08 lowercase basename
//
void listLfn(SdBaseFile* dirFile) {
  uint8_t offset[] = {1, 3, 5, 7, 9, 14, 16, 18, 20, 22, 24, 28, 30};
  char name[13];
  char lfn[131];
  bool haveLong = false;
  dir_t dir;
  uint8_t i;
  uint8_t lfnIn = 130;
  uint8_t ndir;
  uint8_t sum;
  uint8_t test;

  dirFile->rewind();
  while (dirFile->read(&dir, 32) == 32) {
    if (DIR_IS_LONG_NAME(&dir)) {
      if (!haveLong) {
        if ((dir.name[0] & 0XE0) != 0X40) continue;
        ndir = dir.name[0] & 0X1F;
        test = dir.creationTimeTenths;
        haveLong = true;
        lfnIn = 130;
        lfn[lfnIn] = 0;
      } else if (dir.name[0] != --ndir || test != dir.creationTimeTenths) {
        haveLong = false;
        continue;
      }
      char *p = (char*)&dir;
      if (lfnIn > 0) {
        lfnIn -= 13;
        for (i = 0; i < 13; i++) {
          lfn[lfnIn + i] = p[offset[i]];
        }
      }
    } else if (DIR_IS_FILE_OR_SUBDIR(&dir)
      && dir.name[0] != DIR_NAME_DELETED
      && dir.name[0] != DIR_NAME_FREE) {
      if (haveLong) {
        for (sum = i = 0; i < 11; i++) {
           sum = (((sum & 1) << 7) | ((sum & 0xfe) >> 1)) + dir.name[i];
        }
        if (sum != test || ndir != 1) haveLong = false;
      }
      SdFile::dirName(dir, name);
      Serial.print(name);
      if (haveLong) {
        Serial.print("  ");
        Serial.print(lfn + lfnIn);
      } else if (dir.reservedNT) {
        Serial.print(" 8.3 lowercase bits: ");
        Serial.print(dir.reservedNT, HEX);
      }
      Serial.println();
      haveLong = false;
    } else {
      if (dir.name[0] == DIR_NAME_FREE) return;
      haveLong = false;
    }
  }
}
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
  if (!sd.begin(SD_CS_PIN)) sd.initErrorHalt();
  // list files in root directory (volume working directory)
  listLfn(sd.vwd());
  Serial.println("Done");
}
void loop() {}

Here is typical output:

LONGFI~1.TXT  Longfilename.txt
SHORT.TXT
NAMEWI~1.TXT  Name with blanks.txt
LOWER.TXT 8.3 lowercase bits: 18
Done

Note that there is no long file name for the entry "LOWER.TXT".  This file has the lower case bits 0X18 set so its lfn is "lower.txt".

Movykappa,

Thanks for the suggested improvement.  Many people have suggested other features and "improvements".

The purpose of this examples is to provide a simple base for fast data logging that can easily be modified so I have not complicated the example with the many suggested additions.

I will likely add a function to SdFat to create a new file with rotation numbers.

The function would be used like this:

Code:

  if (!file.createUnique("LOG.CSV")) {
    Serial.println("createUnique failed");
    while (1);
  }

This would create and open a new empty file with name "LOGnn.CSV" where  nn is 00 - 99.  This would prevent writing over existing files.

The tick is every 1024 microseconds on a 16 MHz CPU.  The delay will be 195 ticks or 199.68 ms.

Edit: 

The Arduino AVR core sets the timer 0 prescale factor to 64. Timer 0 overflows at the frequency of F_CPU/16384L.

The tick for FreeRTOS and millis() happens every 1000000*16384/F_CPU microseconds.

Note that millis() doses not advance every millisecond.  For a 16 MHz cpu millis() advances every 1024 microseconds.  The counter for millis() advances by two every 41 or 42 ticks.

Here is the code for the timer 0 overflow that occasionally bumps m by an extra count:

Code:

m += MILLIS_INC;
f += FRACT_INC;
if (f >= FRACT_MAX) {
f -= FRACT_MAX;
m += 1;
}

You need to set D10 (SS) as an output for the SD library to work correctly.

This is not true, SdFat makes SS an output and set it high.  You must not confuse SS, Slave Select, and Chip Select which are often different pins.

The requirement is to put the chip select pin for all other SPI devices in output mode and set these pins high.

For a standard Ethernet Shield the W5100 CS pin is pin 10.  You must set this pin high if you are not using a 328 Arduino.

To repeat:

*Where* does it fail?  What does the sketch output to the serial monitor (if anything)?  How do you know it actually failed?

SD.h is just a wrapper for an old version of my SdFfat library so that is what you are using in both cases.

card.init(2, csPin) runs the card at 1/4 speed.  You have an SPI problem, maybe your wiring is not clean enough to run at high speed.

SD errorCode: 0X10
SD errorData: 0X1

This is a read error for one of the SD card's info registers.  This is almost always due to problems with the SPI bus.

The standard SD.h wrapper does not allow you to set the SPI bus speed, it always runs at 1/2 speed.  New versions of SdFat allow the bus speed to be set with the second argument of sd.begin(csPin, speed).

You will be "corrupting your cards" unless you fix the SPI problem or run the SPI bus at very slow speeds.

Here is a function that will read a file chunk into any type destination.

Code:

#include <SdFat.h>
SdFat sd;
SdFile file;
const uint8_t SD_CS = SS;
uint32_t bgnBlock;
uint32_t endBlock;

uint16_t wave[1024];
//--------------------------------------------------------
bool readChunk(void* buf, uint32_t startBlock, uint16_t blockCount) {
  uint8_t* dst = (uint8_t*)buf;
  if (!sd.card()->readStart(startBlock)) return false;
  for (uint16_t i = 0; i < blockCount; i++) {
    if (!sd.card()->readData(dst + i*512L)) return false;
  }
  return sd.card()->readStop();
}
//---------------------------------------------------------
void setup() {
  Serial.begin(9600);
  if (!sd.begin(SD_CS) || !file.open("TEST.WAV", O_READ)) {
      Serial.println("begin/open");
      while(1);
  }
  if (!file.contiguousRange(&bgnBlock, &endBlock)) {
    Serial.println("not contiguous");
    while(1);
  }
  uint16_t n = 4;  
  uint32_t t0 = micros();

  if (!readChunk(wave, bgnBlock, n)) {
    Serial.println("readChunk");
    while(1);
  }
  uint32_t t = micros() - t0;
  Serial.print("blocks: ");
  Serial.println(n);
  Serial.print("micros: ");
  Serial.println(t);
  Serial.print("MB/sec: ");
  Serial.println(512.0*n/t);
}
void loop() {}

Here is timing for reading a 1024 element array of uint16_t.

blocks: 4
micros: 576
MB/sec: 3.56

Here is a sketch that will read a file using raw reads.

I tested it with about a five MB file on a 1 GB ATP industrial SD.

The result was a read speed of about 4.5 MB/sec on Due.

blocks: 9765
micros: 1094559
MB/sec: 4.57

Code:

#include <SdFat.h>
SdFat sd;
SdFile file;
static const uint8_t SD_CS = SS;
uint32_t bgnBlock;
uint32_t endBlock;
uint8_t buf[512];

void setup() {
  Serial.begin(9600);
  if (!sd.begin(SD_CS) || !file.open("TEST.WAV", O_READ)) {
      Serial.println("begin/open");
      while(1);
  }
  if (!file.contiguousRange(&bgnBlock, &endBlock)) {
    Serial.println("not contiguous");
    while(1);
  }
  // count of blocks in file;
  uint32_t n = (file.fileSize() + 511)/512;
  // read start time
  uint32_t t0 = micros();
  
  // address of first block
  sd.card()->readStart(bgnBlock);

  for (uint32_t i = 0; i < n; i++) {
    if (!sd.card()->readData(buf)) {
      Serial.println("readBlock");
      while(1);
    }
  }
  sd.card()->readStop();
  uint32_t t = micros() - t0;
  Serial.print("blocks: ");
  Serial.println(n);
  Serial.print("micros: ");
  Serial.println(t);
  Serial.print("MB/sec: ");
  Serial.println(512.0*n/t);
}
void loop() {}

Edit: I did some tests with four and eight block reads to get the time to read a chunk of a file.  These are using the industrial ATP card so will be faster than some consumer cards.

blocks: 4
micros: 574
MB/sec: 3.57

blocks: 8
micros: 1022
MB/sec: 4.01

So you can read a 2048 byte chunk in 574 usec and a 4096 byte chunk in 1022 usec.

Coding Badly is right, the only way to insure low noise due to ADC sample time jitter is to auto trigger the ADC with Timer/Counter1 Compare Match B.

At the rate chaveiro is sampling, 1600 samples per second, sub microsecond jitter is needed to avoid jitter noise in the AVR 10 bit ADC.  The jitter SNR must be greater than 62 dB to get 10 bit ADC accuracy.

The formula for ADC jitter noise is

SNR = 20*log(1/(2*pi*f*t))

where f is the sample frequency and t is the jitter time.

with 0.1 usec jitter the SNR is about 60, just about good enough.

SNR = 20*log10(1/(6.28*1600*1e-7)) = 59.96

Here is a calculator for ADC jitter http://www.maximintegrated.com/design/tools/calculators/general-engineering/jitter.cfm

It gives 9.71e-8 seconds max jitter for full accuracy of a 10-bit ADC at 1600 samples per second.

The SdFile object acts like a file handle in other systems.  It contains information from the directory entry and cluster information for the current position.  A number of blocks must be read from file structures to open a file and seek to a position.

If I wanted to optimize reads from a large number of files I would use raw SD reads.

When you copy files to freshly formatted SD, the files are contiguous.  SdFat has a function to determine if a file is contiguous and where the blocks are located.

bool SdBaseFile::contiguousRange    (    uint32_t *     bgnBlock,
      uint32_t *     endBlock
   )       

Check for contiguous file and return its raw block range.

Parameters:
    [out]   bgnBlock   the first block address for the file.
    [out]   endBlock   the last block address for the file.

Returns:
    The value one, true, is returned for success and the value zero, false, is returned for failure. Reasons for failure include file is not contiguous, file has zero length or an I/O error occurred.

I would open each file and find its location with the the above function.

I would then use either the Sd2Card single block read function:

Code:

bool Sd2Card::readBlock (uint32_t block, uint8_t *dst);

Or the Sd2Card multi-block sequence:

Code:

bool Sd2Card::readStart (uint32_t blockNumber);  // set start block for a multiple block read sequence.

bool Sd2Card::readData (uint8_t *  dst);  // Read one data block in a multiple block read sequence.

bool Sd2Card::readStop ();  // End a read multiple blocks sequence.

SD cards do look ahead for multiple block reads so are very efficient in this mode.

I understand you going to add RAM but I am curious why reading from the SD fails.

I looked at the Groovuino library and was astounded that only one file handle was uses and files are opened and closed while playing sound.

Opening a file is very slow so I would have used an array of file handles and opened all the files before playing sound.  A file handle only requires about 32 bytes.  Rewinding or seeking to the start of a file requires no SD access for an open file.

Buffers sizes that are a power of two likely will work best.  The cluster size for files is a power of two and reading across a cluster boundary will defeat the larger buffer size.  Try 4 KB buffers.

Free RAM: 91423
Type is FAT32
File size 10MB
Buffer size 4096 bytes

Starting read test.  Please wait up to a minute
Read 2848.53 KB/sec
Maximum latency: 3510 usec, Minimum Latency: 1339 usec, Avg Latency: 1436 usec

Make sure you SD is formatted correctly with 32 KB clusters.  SD formatter https://www.sdcard.org/downloads/formatter_4/  places file system structures for best performance.