Problem data logging to a contiguous file

Hi,

I have been trying to modify the RawWrite example from the sdFat library to allow me to write data from a real time clock and an imu to an sd card.
The code compiles and runs fine but the data from the struct array is never written to the file.
The code works without the rtc and imu.

I am using:
Arduino Pro Mini 5v ATMega 328
RTC_DS1307
Adafruit_BNO055 imu
Adafruit microSD card breakout board

I have included the code I am running below.

Can anyone help me to get the struct data written to the sd card?
Many Thanks in advance.

#include <Wire.h>
#include "I2Cdev.h"
#include <Adafruit_Sensor.h> 
#include <Adafruit_BNO055.h>
#include <utility/imumaths.h>
#include <Time.h> // rtc
#include "RTClib.h"
#include <SPI.h>
#include "SdFat.h"
#include "sdios.h"

// SD chip select pin
const uint8_t chipSelect = 8;

const uint32_t RATE_KB_PER_SEC = 100;

const uint32_t TEST_TIME_SEC = 100;

// number of blocks in the contiguous file
const uint32_t BLOCK_COUNT = (1000*RATE_KB_PER_SEC*TEST_TIME_SEC + 511)/512;
const uint32_t ERASE_SIZE = 262144L;

uint8_t* pCache;
 
// file system
SdFat sd;

// test file
SdFile file;

// file extent
uint32_t bgnBlock, endBlock;

// Serial output stream
ArduinoOutStream cout(Serial);

int count = 0;

Adafruit_BNO055 bno = Adafruit_BNO055(19, 0x29);
RTC_DS1307 rtc; 

//filename
#define FILE_BASE_NAME "REDT"
const uint8_t BASE_NAME_SIZE = sizeof(FILE_BASE_NAME) - 1;
char fileName[] = FILE_BASE_NAME "00.BIN";

// 48 bytes
typedef struct { 
    uint8_t yr;    
    uint8_t hr;
    uint8_t mn;
    uint8_t s;
    unsigned long ms;
    float angv_x;
    float angv_y;
    float angv_z;
    float acc_x;
    float acc_y;
    float acc_z;
    float quat_w;
    float quat_x;
    float quat_y;
    float quat_z;
} datastore; 

datastore MyData[10];
//------------------------------------------------------------------------------
// store error strings in flash to save RAM
#define error(s) sd.errorHalt(F(s))
//------------------------------------------------------------------------------

// ================================================================
// ===                    CALIBRATION                           ===
// ================================================================
void displayCalStatus(void)
{
  /* Get the four calibration values (0..3) */
  /* Any sensor data reporting 0 should be ignored, */
  /* 3 means 'fully calibrated" */
  uint8_t system, gyro, accel, mag;
  system = gyro = accel = mag = 0;
  bno.getCalibration(&system, &gyro, &accel, &mag);
  
  while ((system < 3) || (gyro < 3) || (mag < 3))
  {
    system = gyro = accel = mag = 0;
    bno.getCalibration(&system, &gyro, &accel, &mag);
    /* Get a new sensor event */
    sensors_event_t event;
    bno.getEvent(&event);
    /* Display the individual values */
    Serial.print("X: ");
    Serial.print(event.orientation.x, 4);
    Serial.print("\tY: ");
    Serial.print(event.orientation.y, 4);
    Serial.print("\tZ: ");
    Serial.print(event.orientation.z, 4);
    Serial.print("\tSys:");
    Serial.print(system, DEC);
    Serial.print(" G:");
    Serial.print(gyro, DEC);
    Serial.print(" A:");
    Serial.print(accel, DEC);
    Serial.print(" M:");
    Serial.print(mag, DEC);
    Serial.print("\n");
    digitalWrite(5, HIGH);   // LED on 
    delay(1000);              // wait for a second
    digitalWrite(5, LOW);    //led off
    delay(1000);
  }
  // Long blink signals successful calibration
  digitalWrite(5, HIGH);   // LED on 
  delay(5000);              // wait for 5 seconds
  digitalWrite(5, LOW);    //led off
  Serial.println("CALIBRATED!");
}



// ================================================================
// ===               SET TIMESTAMP OF FILE                      ===
// ================================================================
void dateTime(uint16_t* date, uint16_t* time) {
   DateTime now = rtc.now();

  // return date using FAT_DATE macro to format fields
  *date = FAT_DATE(now.year(), now.month(), now.day());

  // return time using FAT_TIME macro to format fields
  *time = FAT_TIME(now.hour(), now.minute(), now.second());
}

void setup(void) {
  Serial.begin(9600);
  
  // Wait for USB Serial 
  while (!Serial) {
    SysCall::yield();
  }

  /* Initialise the sensor */
  if(!bno.begin())
  {
    /* There was a problem detecting the BNO055 ... check your connections */
    error("No BNO055 detected");
  }
  
  delay(1000);

  bno.setExtCrystalUse(true);

  pinMode(5, OUTPUT); //led
  pinMode(10, OUTPUT); // bno
  pinMode(chipSelect, OUTPUT); //sd
  
  // Initialize at the highest speed supported by the board that is
  // not over 50 MHz. Try a lower speed if SPI errors occur.
  if (!sd.begin(chipSelect, SD_SCK_MHZ(50))) {
    sd.initErrorHalt();
  }

  // delete possible existing file
  sd.remove("RawWrite.bin");
  sd.remove("REDT.BIN");
  
  //Calibration status values: 0=uncalibrated, 3=fully calibrated"
  //displayCalStatus();
  
   while (sd.exists(fileName)) {
     if (fileName[BASE_NAME_SIZE + 1] != '9') {
       fileName[BASE_NAME_SIZE + 1]++;
     } else if (fileName[BASE_NAME_SIZE] != '9') {
       fileName[BASE_NAME_SIZE + 1] = '0';
       fileName[BASE_NAME_SIZE]++;
       Serial.println(fileName);
     } else {
       error("Can't create file name");
    }
  }


  SdFile::dateTimeCallback(dateTime);

  // create a contiguous file
  if (!file.createContiguous("RawWrite.bin", 512UL*BLOCK_COUNT)) {
    error("createContiguous failed");
  }
  // get the location of the file's blocks
  if (!file.contiguousRange(&bgnBlock, &endBlock)) {
    error("contiguousRange failed");
  }

  if(!file.rename("REDT.BIN")){
    error("rename failed");
  }

  //*********************NOTE**************************************
  // NO SdFile calls are allowed while cache is used for raw writes
  //***************************************************************

  // clear the cache and use it as a 512 byte buffer
  pCache = (uint8_t*)sd.vol()->cacheClear();

  // fill cache with eight lines of 64 bytes each
  memset(pCache, '0', 512);

  // flash erase all data in file
  uint32_t bgnErase = bgnBlock;
  uint32_t endErase;
  while (bgnErase < endBlock) {
    endErase = bgnErase + ERASE_SIZE;
    if (endErase > endBlock) endErase = endBlock;
    if (!sd.card()->erase(bgnErase, endErase)) ;
    bgnErase = endErase + 1;
  }

   // tell card to setup for multiple block write with pre-erase
  if (!sd.card()->writeStart(bgnBlock, BLOCK_COUNT)) {
    error("writeStart failed");
  }
}
//------------------------------------------------------------------------------
void loop(void) {
 
  // Read any extra Serial data.
  do {
    delay(10);
  } while (Serial.available() && Serial.read() >= 0);
  // F stores strings in flash to save RAM
  cout << F("Type any character to start\n");
  while (!Serial.available()) {
    SysCall::yield();
  }


  delay(1000);
  // write data
  MyData[count].yr = 19;
  
  DateTime now = rtc.now();
  MyData[count].hr = now.hour();
  MyData[count].mn = now.minute();
  MyData[count].s = now.second();
  MyData[count].ms = millis();
  
  imu::Vector<3> gyros = bno.getVector(Adafruit_BNO055::VECTOR_GYROSCOPE);
  MyData[count].angv_x = gyros.x();
  MyData[count].angv_y = gyros.y();
  MyData[count].angv_z = gyros.z();
  Serial.println(MyData[count].angv_z);

  imu::Vector<3> accel = bno.getVector(Adafruit_BNO055::VECTOR_ACCELEROMETER);
  MyData[count].acc_x = accel.x();
  MyData[count].acc_y = accel.y();
  MyData[count].acc_z = accel.z();
 
  imu::Quaternion quat = bno.getQuat();
  MyData[count].quat_w = quat.w();
  MyData[count].quat_x = quat.x();
  MyData[count].quat_y = quat.y();
  MyData[count].quat_z = quat.z();
  Serial.println(MyData[count].quat_z);

  count++;
  Serial.println(count);
  
  if (count == 10){

    if(!memcpy(pCache, &MyData, sizeof(MyData))){
      error("memcpy failed");
    }

    if (!sd.card()->writeData(pCache)) { 
      error("writeData failed");
    }
  
    count = 0; 
    memset(pCache, ' ', 512); 
   

  Serial.println();
  }
}

What does the code actually do?

*date = ...

This is a copy of a pointer, changing it won't work.

Hi PaulS

When I run the code the values from the RTC and the BNO055 sensor are written to the struct. The contiguous file is created. The code runs without giving an error message but the file on the sd card just contains zeros.

I have been trying to modify the RawWrite example from the sdFat library to allow me to write data from a real time clock and an imu to an sd card.

I don't really understand this statement in context with code riddled with delay()s. The whole reason for using RawWrite is speed. High speed writing of data, now and then, makes not a lick of sense.

When I run the code the values from the RTC and the BNO055 sensor are written to the struct.

You know that because?

You are theoretically storing data in an array of structs. While I have no real reason to doubt that that is happening, I would make CERTAIN that that actually happened, before assuming that copying the data to some pointed to location would be successful/useful.

I'm not familiar with the example, or the SdFat library, but I have to wonder why you set a pointer to point to some memory that the SdFat library owns, but then need to tell the library where the data to write to the file lives. Maybe that is reasonable. Maybe not.

What I would do is use malloc() to reserve some space, copy the data to that space, and then tell the writeData() method where the data was. But, then, that might not really be necessary.

In debugging, the very first thing to do is to assure yourself that the array of structs IS valued correctly.

Many thanks for your reply PaulS. That is really helpful.

I will definitely remove the delays.

I have previously used the same struct to write to a binary file and it was successful. In this iteration the change I have made is to add the code from the sdfat rawWrite example.

Is there another way to test that the data is being written to the struct other than the Serial.println statements?

I will have a look at malloc as well.

I wonder whether it might be a problem with the SD Card being fat32 formatted?

Is there another way to test that the data is being written to the struct other than the Serial.println statements?

I'll answer your question with a question, though that is generally not considered polite. Just what is wrong with that approach? Long term, after you KNOW that the code is working, you don't need to do that. But for debugging purposes, that is the best approach.

I wonder whether it might be a problem with the SD Card being fat32 formatted?

Try some other examples from the SdFat library, to test that hypothesis.