Time to record 1 analog datastream using I2C and SD card logger?

HI all,

I’m trying to use Adafruit’s 12-bit I2C ADC (http://www.adafruit.com/products/1083) with Adafruit’s SD-card datalogger (Overview | Adafruit Data Logger Shield | Adafruit Learning System), and I need a high-frequency record (1000Hz) of an analog force signal. However, the problem is that it seems to take a really long time to get each datapoint. Hoping someone can help. The idea is to have an interval of 1 milisecond between readings and take reading for 1 second (i.e. 1000 samples). However, when I run the code as is, even though the delay is set to 1 milisecond, it takes much more than 1 second to gather the 1000 datapoints (sometimes upwards of 20 seconds).

Any ideas on how to speed up the code? What did I do wrong?

Thanks so much.

–Nick

// Date and time functions using a DS1307 RTC connected via I2C and Wire lib

#include <Wire.h>
#include "RTClib.h"
#include "SPI.h"
#include "SD.h"


#define forcePin 0           // Defines force as thatever analog signal foes into analog 0
#define LOG_INTERVAL 1 // mills between entries
#define N_Samples 1000 // mills between entries
#define ECHO_TO_SERIAL 0 // echo data to serial port
#define WAIT_TO_START 1 // Wait for serial input in setup()

RTC_DS1307 RTC;


// for the data logging shield, we use digital pin 10 for the SD cs line
const int chipSelect = 10; // this dictates the chip selcted for the SD card.

int force_address = 72;

//// the logging file
File logfile;
//

void setup () {
  Serial.begin(57600);
  Wire.begin();
  RTC.begin();

#if WAIT_TO_START
  Serial.println("Type any character to start");
  while (!Serial.available());
#endif //WAIT_TO_START

  if (! RTC.isrunning()) {
    Serial.println("RTC is NOT running!");
    // following line sets the RTC to the date & time this sketch was compiled
    // uncomment it & upload to set the time, date and start run the RTC!
    //RTC.adjust(DateTime(__DATE__, __TIME__));
  }

  // initialize the SD card
  Serial.print("Initializing SD card...");
  // make sure that the default chip select pin is set to
  // output, even if you don't use it:
  pinMode(10, OUTPUT);
  digitalWrite(10, HIGH); 

  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    // don't do anything more:
    return;

  }


  Serial.println("card initialized.");
  // create a new file
  char filename[] = "LOGGER00.CSV";
  for (uint8_t i = 0; i < 100; i++) {
    filename[6] = i/10 + '0';
    filename[7] = i%10 + '0';
    if (! SD.exists(filename)) {
      // only open a new file if it doesn't exist
      logfile = SD.open(filename, FILE_WRITE);
      break; // leave the loop!
    }
  }

  Serial.print("Logging to: ");
  Serial.println(filename);


  logfile.println("millis,force,timepoint"); 

}

void loop () {

  for (int i=0; i <= N_Samples; i++){


    DateTime now = RTC.now();

    delay(LOG_INTERVAL);   // delay for the amount of time we want between readings

    Wire.beginTransmission(force_address);
    Wire.write(0);
    Wire.endTransmission();
    Wire.requestFrom(force_address, 1);
    while(Wire.available() == 0);
    int temp_force = Wire.read();




    // log milliseconds since starting
    uint32_t m = millis();
    logfile.print(m); // milliseconds since start
    logfile.print(", ");

#if ECHO_TO_SERIAL
    Serial.print(m); // milliseconds since start
    Serial.print(", ");
    Serial.println();
#endif

    logfile.print(temp_force);
    logfile.print(", ");
    logfile.print(i);
    logfile.println();

#if ECHO_TO_SERIAL
    Serial.print(", ");
    Serial.print(temp_force);
    Serial.print(", ");
    Serial.println();
#endif //ECHO_TO_SERIAL

    logfile.flush();

  }
  Serial.println("Logging finished");
  while(1) { 
  }
}

That seems like it's too fast for the sd card. It takes time to open and write to file.

Maybe save the data to an array and then save it to the sd card but I don't think the arduino has enough memory to save thousands of records.

#define N_Samples 1000 // mills between entries

Well, no wonder it takes so long... Wait, what does that comment have to do with that code?

    int temp_force = Wire.read();

Wire.read() doesn't return an int.

    delay(LOG_INTERVAL);   // delay for the amount of time we want between readings

That's not going to make the readings happen at regular intervals. That's going to provide a consistent gap between the end of one cycle and the start of another. Not the same thing at all.

Mistergreen, that makes perfect sense. I’ve rewritten the code to create an array of just the RTC clock and the force measure, to cut down on the amount of memory that is used.

PaulS, those are all great points! I had not yet attacked the actually-reading -the-analog-signal-via-I2C portion of the code yet. I’ve done that new and it works great.

I do still have a couple of questions/issues:

  1. The memory buffer goes cross-eyed (i.e. starts making corrupt files, etc) if I have it take anything more than about 130 datapoints. Is there any way to increase the onboard memory, or to make the data smaller, to get a few more. I’d be happy with 20 datapoints, and I’d settle for 150+ – I’m so close!

  2. there are 3-4 miliseconds between datapoints even when I completely get rid of the delay, even without writing to the SD card. Is there any way to speed up that section of code (the first “for” loop)? I’d love to be able to at least set the data collection frequency, say every 4ms (250 Hz). Ideally here, I’d love to have at least 500Hz collection frequency, but I’d settle for whatever I can get.

THanks so much! THis has been REALLY helpful

#include <Wire.h>
#include "RTClib.h"
#include "SPI.h"
#include "SD.h"
#include <Adafruit_ADS1015.h>

Adafruit_ADS1015 ads1015;
RTC_DS1307 RTC;

#define LOG_INTERVAL 10 // mills between entries
#define ECHO_TO_SERIAL 0 // echo data to serial port
#define N_Samples 130
#define WAIT_TO_START 1 // Wait for serial input in setup()


// for the data logging shield, we use digital pin 10 for the SD cs line
const int chipSelect = 10; // this dictates the chip selcted for the SD card.



//// the logging file
File logfile;
//

void setup () {
  Serial.begin(57600);
  Wire.begin();
  RTC.begin();
  ads1015.begin(); 

#if WAIT_TO_START
  Serial.println("Type any character to start");
  while (!Serial.available());
#endif //WAIT_TO_START

  if (! RTC.isrunning()) {
    Serial.println("RTC is NOT running!");
    // following line sets the RTC to the date & time this sketch was compiled
    // uncomment it & upload to set the time, date and start run the RTC!
    //RTC.adjust(DateTime(__DATE__, __TIME__));
  }

  // initialize the SD card
  Serial.print("Initializing SD card...");
  // make sure that the default chip select pin is set to
  // output, even if you don't use it:
  pinMode(10, OUTPUT);
  digitalWrite(10, HIGH); 

  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    // don't do anything more:
    return;

  }


  Serial.println("card initialized.");
  // create a new file
  char filename[] = "LOGGER00.CSV";
  for (uint8_t i = 0; i < 100; i++) {
    filename[6] = i/10 + '0';
    filename[7] = i%10 + '0';
    if (! SD.exists(filename)) {
      // only open a new file if it doesn't exist
      logfile = SD.open(filename, FILE_WRITE);
      break; // leave the loop!
    }
  }

  Serial.print("Logging to: ");
  Serial.println(filename);


  logfile.println("RTCtime,Force"); 
  logfile.flush();

}

void loop () {

  int RTCclockArray[N_Samples];
  int forceArray[N_Samples];

  for (int i=0; i <= N_Samples; i++){


    DateTime now = RTC.now();

    //   delay(LOG_INTERVAL);   // delay for the amount of time we want between readings

    int16_t adc0;

    int temp_force = ads1015.readADC_SingleEnded(0);

    // log milliseconds since starting
    uint32_t m = millis();

    RTCclockArray[i] = m;
    forceArray[i] = temp_force;

#if ECHO_TO_SERIAL
    Serial.print("RTC time: ");
    Serial.print(m); // milliseconds since start
    Serial.print(", force: ");
    Serial.println(temp_force);
    Serial.println();
#endif //ECHO_TO_SERIAL
  }

  Serial.println("Data finished. Logging to card.");

  for (int i=0; i <= N_Samples; i++)
  {
    logfile.print(RTCclockArray[i]); // milliseconds since start
    logfile.print(", ");
    logfile.println(forceArray[i]); 

    logfile.flush();
  }
  Serial.println("Logging finished");
  while(1) { 
  }
}

OH! Perhaps I just use an Arduino Due: Arduino - ArduinoBoardDue

Looks like that has a lot more flash memory and faster processing -- might that get me a few more datapoints and a bit better temporal resolution? If so, any guesses as to how much?

Thanks.

--Nick