Go Down

Topic: Occasional slow-down of SD writing from Due (Read 157 times) previous topic - next topic

KCL-Dagorlad

I take ASCII data from an RS232 input and write this to SD card, adding a millis timestamp to each line, project running on a Due.

It works, but the Due and/or SD writing action seem to suffer an occasional significant delay/interruption that can last up to 1 second, and this occurs unpredictably but roughly once every minute.

RS232 input is a string of 38 characters (38 bytes) at 57600 baud, arriving every 0.02 second interval (50Hz), via an RS232-to-TTL breakout board wired to Due. Each string is terminated by a '\n' character (linefeed).

SD output is to an Adafruit Micro-SD breakout board connected via SPI and using SdFat library.

RS232 input and SD output work fine most of the time, BUT every minute or so the writing to SD card seems to get delayed longer and longer, the loop buffer on the RS232 input then maxes out and starts to get overwritten and the whole data transfer goes wrong. Because I add a millis timestamp to each line written on the SD I can see this: normally there are exactly 20 millis between each line, but when it goes wrong suddenly there are ~100 millis time difference to the next line and this is when the RS232 buffers starts to get overwhelmed.

Why is this happening? Is this to do with the SD writing, OR is there something happening on the Due processor that interrupts the execution?

Code: [Select]

#include <SdFat.h>
#define rs232Serial Serial1

SdFat sd;
SdFile dataFile;

char someChar;

void setup() {
  pinMode (19, INPUT); // RS232 RX
  pinMode (18, OUTPUT); // RS232 TX
  pinMode (10, OUTPUT); // chipSelect

  rs232Serial.begin(57600);
  sd.begin(10, SPI_HALF_SPEED); // NB: SD doesn't work at SPI_FULL_SPEED

  dataFile.open("newfile.dat", O_CREAT | O_WRITE);

}

void loop() {

  if (rs232Serial.available() > 0) {
    someChar = rs232Serial.read();
    if (someChar == '\n') {
      dataFile.write(',');
      dataFile.print(millis());
      dataFile.write(someChar);
      dataFile.flush();
    }
    else {
      dataFile.write(someChar);
    }
  }

}


Note that I need to keep the main loop as cheap as possible, because later-on I will need to include a pulse-counter via the external interrupt which will take up a lot of bandwidth.

I am using a class-10 SD card from Sandisk.

The data problem looks like this in the file "newfile.dat" on the micro-SD, comments in //:

Code: [Select]

03,00,+00.00,+00.00,-00.02,+24.80,09,101637    // the last number is the timestamp in milliseconds
04,00,+00.00,+00.00,-00.02,+24.80,0E,101657    //  the stuff in front is the 38-char RS232 coming in every 0.02 s
05,00,+00.00,+00.00,-00.02,+24.80,0F,101677     // we are now 101.6 seconds into the data collection
06,01,+00.00,+00.00,-00.02,+24.80,0D,101697     // 20 millisecond interval from previous
01,10,+00.00,+00.00,-00.02,+24.81,0B,101717    // 20 millisecond interval from previous
02,38,+00.00,+00.00,-00.02,+24.81,02,101737    // 20 millisecond interval from previous
03,00,+00.00,+00.00,-00.02,+24.81,08,101757    // 20 millisecond interval from previous
04,00,+00.00,+00.00,-00.02,+24.81,0F,101777    // 20 millisecond interval from previous
05,00,+00.00,+00.00,-00.02,+24.81,0E,101914    // 137 millisecond interval!! There has been an interruption somehow!
06,01,+00.00,+00.00,-00.02,+24.81,0C,101972    // 58 millisecond interval... more interruption
01,10,+00.00,+00.00,-00.02,+24.81,0B,102039    // 67 millisecond interval... still further delay
02,38,+00.00.02,+24.81,0E,102103                    // at this point the RS232 loop buffer has overflowed with new incoming data
06,01,+00.00,+00.00,.00,-00.02,+24.81,02,102217          // losing linefeed markers so output goes haywire
03,00,+00.00,+006,01,+00.00,+00.00,-00.02,+24.81,0C+03,00,+00.00,+00.00,-00.027,102226
03,00,+00.00,+00.00,-00.02,+24.83,0A,102237
04,00,-00.01,+00.01,-00.02,+24.83,0B,102257    // 20 millisecond interval from previous, system has recovered
05,00,-00.01,+00.01,-00.02,+24.83,0A,102277    // 20 millisecond interval from previous
06,01,+00.00,+00.00,-00.02,+24.83,0E,102297    // ...
01,10,+00.00,+00.00,-00.02,+24.83,09,102317
02,38,-00.01,+00.01,-00.02,+24.83,06,102337
03,00,-00.01,+00.01,-00.02,+24.83,0C,102357
04,00,-00.01,+00.01,-00.02,+24.83,0B,102377
05,00,-00.01,+00.01,-00.02,+24.83,0A,102397


The incoming RS232 is tested and confirmed to output the 38-char string every 0.02 seconds reliably, so this is not the source of the delays seen in the file output.

These slow-downs are not at an exact/regular period, so I can't understand why this is happening, it doesn't seem to relate to the data input/output itself...

Does anyone have any ideas?

ard_newbie

Maybe this thread will answer your question :



http://forum.arduino.cc/index.php?topic=323421.0

A workaround consist of writing in SRAM (94 KBytes), and later store on the SD card. Another one would be to write on an FRAM , e.g. https://www.adafruit.com/product/1895

KCL-Dagorlad

Thanks ard_newbie, this is an eye opener, I had not come across this particular post in my previous searches.

?Do you/anyone know: if I channel the RS232 data stream to a 94 Kb SRAM or FLASH store and then write this as a single block to the SD card every time it's full, will the Due be able to keep up with the RS232 input (40 bytes every 0.02 s), while the data is being sent/written to the SD card?

The data I'm handling amounts to 2000 bytes per second, which fills up the 94 Kb memory storage in ~48 seconds.  I need to be able to record this data stream onto the SD card for durations of hours, at an exact and reliable interval of 0.02 seconds...

Go Up