SdFat : Corrupted big files (and FAT chains) enabling separate fat cache

Hi,

I'm developing an audio recorder (.wav format, stereo, 44100 samples/sec, 16 bit/sample), which is recording on an SD card using SdFat. The library is quite fast, I must skip some samples because of card "pausing" sometimes, but that's not a big concern.
The board I'm using is a custom one, using a PIC32MX controller; I added some stuffs in config area and it works ok, besides a corruption of big files when I enable separate fat cache :

#if defined(__arm__) || defined(_FISHINO32_)
//#define USE_SEPARATE_FAT_CACHE 1
#define USE_SEPARATE_FAT_CACHE 0
#else  // __arm__
#define USE_SEPARATE_FAT_CACHE 0
#endif  // __arm__

If I decomment the
#define USE_SEPARATE_FAT_CACHE 1
the files shows some 8 MBytes of data using dir command, but is not accessible; checking with fsck.fat linux app it tells that the clusters are bad and the file must be truncated to around 5 MBytes.
I tried many times; if I don't correct the error, the file is no more writeabile.
I don't think that's related to my setup, as removing the #define all is ok.
It's also not a RAM problem, I've around 80-100 KBytes free.

Try running this test program. I use it on ARM processors as a write/read test. It will write an 8 MB (8,000,000 byte) and check the file's data content.

I ran it on a Due, which uses a separate FAT cache, with this result:

type 'w' to write or 'r' to read
SdFat version 20160719
Starting file write
....................
Write done, rerun program to check file
type 'w' to write or 'r' to read
SdFat version 20160719
Starting file read
....................
Read success, check file on PC

The SD has no errors when checked on a PC.

#include "SPI.h"
#include "SdFat.h"

#define CS_PIN SS
#define FILE_SIZE 8000000
#define ARRAY_DIM 100
#define FILE_NAME "fattest.bin"

SdFat sd;

SdFile file;

uint32_t array[ARRAY_DIM];
#define RDWR_COUNT (FILE_SIZE/sizeof(array))

void readFile() {
  Serial.println("Starting file read");  
  uint32_t n = 0;

  if (!file.open(FILE_NAME, O_READ)) {
    sd.errorHalt("open for read failed");
  }
  for (uint32_t i = 0; i < RDWR_COUNT; i++) {
    if (file.read(array, sizeof(array)) != sizeof(array)) {
      Serial.println();
      sd.errorHalt("read failed");
    }
    for (uint32_t j = 0; j < ARRAY_DIM; j++) {
      if (array[j] != n++) {
        Serial.println();
        sd.errorHalt("data compare error");
      }
    }
    if (i%(RDWR_COUNT/20) == 0) {
      Serial.print('.');
    }    
  }
  Serial.println();
  if (!file.close()) {
    sd.errorHalt("close failed for read");
  }
  Serial.println("Read success, check file on PC");
}

void writeFile() {
  Serial.println("Starting file write");
  uint32_t n = 0;
  if (!file.open(FILE_NAME, O_WRITE | O_CREAT)) {
    sd.errorHalt("open for write failed");
  }
  for (uint32_t i = 0; i < RDWR_COUNT; i++) {
    for (uint32_t j = 0; j < ARRAY_DIM; j++) {
      array[j] = n++;
    }
    if (file.write(array, sizeof(array)) != sizeof(array)) {
      Serial.println();
      sd.errorHalt("write failed");
    }
    if (i%(RDWR_COUNT/20) == 0) {
      Serial.print('.');
    }    
  }
  Serial.println();
  if (!file.close()) {
    sd.errorHalt("close failed for write");
  }
  Serial.println("Write done, rerun program to check file");
}

void setup() {
  char c;
  Serial.begin(9600);
  while (!Serial) {}
  while(1) {
    do {
      delay(10);
    } while (Serial.read() >= 0);
    Serial.println("type 'w' to write or 'r' to read");
    while (!Serial.available()) {}
    c = Serial.read();
    if (c == 'r' || c == 'w') {
      break;
    }
    Serial.print("Invalid input: ");
    Serial.println(c);
    delay(500);
  }
  Serial.print("SdFat version ");
  Serial.println(SD_FAT_VERSION); 
  if (!sd.begin(CS_PIN)) {
    sd.initErrorHalt();
  }
  if (c == 'r') {
    readFile();
  } else {  
    writeFile();
  }
}
void loop() {
  // put your main code here, to run repeatedly:
}

uhmmm... I'll test it later on, but my problem is still there :slight_smile:

It's possible that's a problem triggered by fast interrupts from I2S ? When writing stereo .wav files I've some 44100 x 2 interrupts/second. The interrupt routine do not use SPI, of course, nor do any file I/O, just do some buffering of data coming from an I2S dedicated secondary interface.
The file write is done also through an interrupt routine, with much lower priority (called on buffer full, so some 250 call/second max), which IS interrupted by former one.
When SD lags I just skip samples, no re-entering on any interrupt routine can happen.
No other code is doing file I/O.

You are your own since you don't want to run the test program.

I have recorded audio on an Uno with 44,100 interrupts per second. I don't even have dropped data.

The SdFat AnalogBinLogger works with up to 50,000 interrupts per sec.

Good luck finding your problem.

Let's clarify ; it' NOT that I don't want to run your test program I'm on holydays and I simply can't do right now. I saw your answer and I did what I could at the moment : add some more info. Anyways, I'm usually on my own solving bugs, so this one will be just one more time for me, no problem.About the samples skipping, AFAIK some lag sometimes is foreseen on sd cards, and can be of many tenths of milliseconds. Maybe you've got a non-lagging card; mine is an old one and does lag sometimes, which means to have to drop samples even with an 8k buffer as mine.
Best regards
Max