Performance Optimization (Number of Loops per Second)

Using Arduino Programming Questions
1

I've built a datalogger that logs time and acceleration to an SD card. I am using an Arduino Nano, an Adafruit ADXL375 Breakout, Adafruit DS3231 RTC Breakout, and Adafruit MicroSD Breakout.

Here is my current code:

// I2C Communication
#include <Wire.h>
// Talk to Accelerometer
#include <Adafruit_Sensor.h>
#include <Adafruit_ADXL375.h>
// Talk to RTC
#include <DS3231.h>
// Talk to DataLogger
#include <SD.h>
#include <SPI.h>

// Function Prototype
void deleteAllFilesInDirectory(File dir);

// SD DataLogging Variables
File myFile;
int pinCS = 10;  // CS pin for SD card
// We'll use these two variables to track time since last save
unsigned long startTime;
unsigned long fileTimer;

// Accelerometer Variables
Adafruit_ADXL375 accel = Adafruit_ADXL375(12345); //Assign a unique ID to this sensor at the same time

// RTC Variables
RTClib myRTC;

void setup() {
  // Set Up Serial Monitor
  Serial.begin(9600);
  // Set Up I2C Communication
  Wire.begin();
  Wire.setSpeed(400000);  // Set I2C speed to 400 kHz


  // Set Up Accelerometer
  if(!accel.begin())
  {
    Serial.println("Ooops, no ADXL375 detected ... Check your wiring!");
    while(1);
  }
  accel.setDataRate(ADXL343_DATARATE_3200_HZ);

  // Set Up RTC
  /* Nothing to do, make sure time is already set*/

  // Set Up SD Card
  if (!SD.begin(pinCS)) {
    Serial.println("SD Card Initialization Failed");
    while (1);
  }

  deleteAllFilesInDirectory(SD.open("/"));

  myFile = SD.open("acc_data.csv", FILE_WRITE);
  if (myFile) {
    myFile.println("timestamp,acc_x,acc_y,acc_z");
  }

  startTime = millis();
  fileTimer = millis();
}

void loop() {
  // put your main code here, to run repeatedly:
  // 1) Get Timestamp
  DateTime now = myRTC.now();
  // 2) Get Acceleration
  sensors_event_t event;
  accel.getEvent(&event);
  // 3) Write to File

  if (myFile) {
    myFile.print(now.unixtime());
    myFile.print(",");
    myFile.print(event.acceleration.x);
    myFile.print(",");
    myFile.print(event.acceleration.y);
    myFile.print(",");
    myFile.println(event.acceleration.z);
  }
  // 4) Every 'periodic_save_frequency' minutes save file
  if (millis() - fileTimer >= 60000) { // How many milliseconds to wait before each save
    if (myFile) {
      myFile.close();
    }
    myFile = SD.open("acc_data.csv", FILE_WRITE);
    fileTimer = millis();
    Serial.println("FILE SAVED");
  }
}

void deleteAllFilesInDirectory(File dir) {
  while (true) {
    File entry = dir.openNextFile();
    if (!entry) {
      // no more files
      break;
    }
    if (entry.isDirectory()) {
      // Recursive call to delete files in subdirectory
      deleteAllFilesInDirectory(entry);
      // Remove the empty directory
      SD.rmdir(entry.name());
    } else {
      // Delete the file
      SD.remove(entry.name());
    }
    entry.close();
  }
}

With the way my code is currently written I achieve 200 datapoints per second written to the SD card. I would like to maximize this and ideally hit 800 datapoints per second.

I've already learnt opening and closing the file on the SD card is expensive, which is why I do it periodically. (When I am done testing, I will likely push this to an hour instead of one minute.)

I am new to Arduino / C++ so any help here is appreciated.

I'm guessing the biggest help will be storing the acceleration data in memory of the Arduino Nano and then writing a bunch of data at once to the SD card before running out of memory. I haven't worked on such low level programming, so I'm unclear on how I would implement that.

Any advice is appreciated :slight_smile:

2

I suspect this is already happening. The SD card library will be using a memory buffer, and all the myFile.print() will write that buffer until it gets full, at which point it gets written to the card.

I recommend you experiment to figure out what is the bottleneck preventing the code exceeding that 200 data points per second. Is it reading the RTC? Is it reading the sensor? Is it formatting the data? Is it writing to the card? What else could it be?

The way you can test each of these possible bottlenecks is to remove them one at a time and measure how many datapoints per second are achieved.

3

I think this code will delete an old file and open a new with same name, loosing the whole old data. Is that what you expected?

4

I think this opens the file for append and does not loose the contents.

6
1 Like
7

Simple thing, increase the serial bit-rate so the program doesn’t slow down when the tx buffer fills up.

Also, buffering the SD output data is a worthwhile idea, you should see a significant boost in performance,

1 Like
8

this can be accelerated two to three times if you do not print the data to a file (converting it to text), but write it in binary form

9

nope this doesn't delete the old file, works as expected

10

are there any code samples that could show me how to do this?

11 
myFile.print(',');

Should also be faster than with double quotes...

12

you have at least following bottlenecks

  1. reading the RTC with every loop cycle
  2. reading an I2C sensor
  3. writing to the SD

regarding (1) it should be sufficient if you just get a new timestamp every 0.1 - 0.5 seconds. The resolution of the unix time is anyway only 1 second.
regarding (2) have you tried to use SPI?
regarding (3) I don't think you can gather a lot by combining several datapoints. In the end you must write each datapoint to the SD. I would check if a microcontroller with an internal Filesystem support (like the ESP8266/ESP32 familiy) makes writing to a file faster. Or consider to use a FRAM module.