Nano 33 ble. Logging different sensor at different frequency.

Using Arduino Storage
#1

Dear all,

I am working on a small data logger. I took inspiration from this topic GPS and IMU logging problems - Programming Questions - Arduino Forum . I have same GPS (NEO 7m sending PVT NAV message at 10 Hz) and then two VL53L0X or occasionally two linear potentiometer.
I would like to log on SD card usign SDfat library all the sensor using different frequency for GPS (10 Hz) and for the sensor (50Hz the vl53l0x and 100 Hz the linear potentiometer). I am able to create the buffers of the sensor but not sure how to send in the correct way to the SD.
I had also a modified version of the lowlatencylogger example to log just the sensor whithout GPS, but there it is using buffer of 512bytes so more than what i would like to send to sd every time (1 GPS samples and 5 or 10 sensor samples).
Morevoer using Nano Ble some library like NeoHWserial or RingBuffer are not compatible.
Anyone has suggestions?

#2

I guess you can put counters and flags at the end of the 100Hz loop. Do the 50Hx job every second trip round and ten trips for the 10Hz job.

#3

I tried already to create an isr flag taking in account lowest frequency (GPS) and create a buffer of the logged analog signal inside a loop until the flag is true. Anyway this not allow to me to log when the loop is ongoing so i cannot have the analog signal when gps is logging for example missing one step each gps log.

Anyone has some example?

Moreover i do not know how to manage to log during SD writing that take his own time and store in a buffer everything is logged in the meanwhile.

#4

You may be having two different problems here. The first is simple matter of acting only on a count, while the second is a matter of saving data at 100Hz. As things are, it sounds like you are reading everything in every loop and subsequently dumping most of it, rather than only reading what you want.

#5

SdFat library is not writing until is own buffer is full (at least from what i understood until now) so i have to fill my buffer anyway with all sensors until sd is sending the data to sd. My problem is how to do it in an efficient way

#6

Probably understood wrong... This implies that, if you have less data to record than the capacity of the buffer, you will never get anything.

It might be time to be more forthcoming with your code, along with an explanation as to why you are using SDFat.

#7

So here my code. Probably the creation of the file can be simplied. All the functions and structures are defined in the UserTypes.h

  /*VERSION 1.0 IMU VL53L0X LOG WITH ARDUINO NANO BLE 33*/

// INCLUDE LIBRARIES
#include <SPI.h>
#include <SdFat.h>
#include "UserTypes.h"


// SET BAUDRATE OF GPS. Need to be configured in u-center program
#define GPS_BAUDRATE 9600   //GPS configured in u-center

// Log file base name.  Must be six characters or less.
#define FILE_BASE_NAME "Data"

// Declare sdfat name
SdFat SD;
// SD chip select pin.  Be sure to disable any other SPI devices such as Enet.
const int chipSelect = SS;  //required for SD reader
// PIN TO PPS OF GPS FOR INTERRUPT
const int tpsInt    = 2;    //timepulse interrupt

// Log file base name.  Must be six characters or less.
const uint32_t FILE_BLOCK_COUNT = 128;
#define FILE_BASE_NAME "Data"
const uint8_t BASE_NAME_SIZE = sizeof(FILE_BASE_NAME) - 1;
char fileName[13] = FILE_BASE_NAME "00.txt";

// Log file.
SdFile logfile;

#define error(msg)

// Variables
uint32_t logTime;
uint32_t lastADCsample;
static const uint32_t ADC_INTERVAL    = 10;  //ms between sensor recordings
static const uint8_t  MAX_ADC_SAMPLES = 100 / ADC_INTERVAL; // evenly-spaced between gps recordings
uint8_t  ADCsamples      = 0;

// Boolean definition
bool tasto = 1;                    // Boolean to start and stop the log
bool prev_tasto;
unsigned int millisAtEpoch = 0;    // time at interrupt
volatile boolean epochFlag;        // Flag for interrupt
volatile boolean noGpsFlag;        // Flag to stop gps during sensor recording

static const int LED = 13;         //SD light control
//
// STRUCTURE FOR NAV_PVT GPS FORMAT DEFINITION
NAV_PVT pvt;


//==============================================================================
// User functions.  Edit writeHeader() and logData() for your requirements.

// Write data header.
void writeHeader() {
  logfile.print(F("micros"));
  logfile.print(F(",xSuspF,latitude,longitude,altitude,gSpeed,sAcc"));
  logfile.println();
}

//------------------------------------------------------------------------------
// Create a new preallocated txt file and then record on it

void logData() {
  createTxtFile();
  writeHeader();
  recordFile();
}

void setup() {

while (!Serial) {
  SysCall::yield();    // DO NOTHING UNTIL SWITCH IS TURNED ON
}

  if (!SD.begin(chipSelect, SD_SCK_MHZ(50))) {
SD.initErrorHalt();   // INIT SD CARD
  }

  // BLINK LED 13 UNTIL SETUP IS FINISHED
  if (!SD.begin(chipSelect)) {
while (true) {
  digitalWrite(LED, HIGH);
  delay(75);
  digitalWrite(LED, LOW);
  delay(75);
}
  }

  // DEFINE PIN FOR INPUT AND OUTPUT
  pinMode(7, INPUT);
  pinMode(6, OUTPUT);

  userSetup() ;

  // DEFINE PIN COMING FROM GPS PPS
  attachInterrupt(digitalPinToInterrupt(tpsInt), isr, RISING); //Interrupt for GPS time pulse

}

void loop() {

  /*// DEBUG ONLY
while (Serial1.available())
  {
    char c = Serial1.read();
    Serial.write(c); // uncomment this line if you want to see the GPS data flowing

  } */

  // READ INPUT BUTTON BEFORE TO START LOGGING
  tasto = digitalRead(7);

  if (!prev_tasto && tasto) {
prev_tasto = tasto;
digitalWrite(6, HIGH); //On shows GPS is recording

// START LOGGING ROUTINE
logData();


  }


}

//-----------------------------------------------------------------------------
// CREATE PREALLOCATED 8KB FILE ON THE SD CARD

void createTxtFile() {
  // max number of blocks to erase per erase call
  const uint32_t ERASE_SIZE = 262144L;
  uint32_t bgnBlock, endBlock;

  // Find an unused file name.
  if (BASE_NAME_SIZE > 6) {
error("FILE_BASE_NAME too long");
  }
  while (SD.exists(fileName)) {
if (fileName[BASE_NAME_SIZE + 1] != '9') {
  fileName[BASE_NAME_SIZE + 1]++;
} else if (fileName[BASE_NAME_SIZE] != '9') {
  fileName[BASE_NAME_SIZE + 1] = '0';
  fileName[BASE_NAME_SIZE]++;
} else {
  error("Can't create file name");
}
  }


  // Create new file.
  Serial.println(F("\nCreating new file"));
  logfile.close();
  if (!logfile.createContiguous(fileName, 512 * FILE_BLOCK_COUNT)) {
error("createContiguous failed");
  }
  // Get the address of the file on the SD.
  if (!logfile.contiguousRange(&bgnBlock, &endBlock)) {
error("contiguousRange failed");
  }
  // Flash erase all data in the file.
  Serial.println(F("Erasing all data"));
  uint32_t bgnErase = bgnBlock;
  uint32_t endErase;
  while (bgnErase < endBlock) {
endErase = bgnErase + ERASE_SIZE;
if (endErase > endBlock) {
  endErase = endBlock;
}
if (!SD.card()->erase(bgnErase, endErase)) {
  error("erase failed");
}
bgnErase = endErase + 1;
  }
}


  
//------------------------------------------------------------------------------
// RECORD FILE ROUTINE (MISSING TRUNCATING AT THE END OF THE FILE

void recordFile() {

  bool closeFile = false;
  Serial.println(F("Logging..."));

  uint32_t logTime = 0 ;
// Check if file is closed
  while (1) {
  logTime += ADC_INTERVAL;

tasto = digitalRead(7);

if (prev_tasto && !tasto) {
  prev_tasto = tasto;
  closeFile = true;
  logfile.close();
  digitalWrite(6, LOW);
  Serial.println(F("File closed"));

  if (closeFile) {
    break;   // RESET BOARD READY FOR NEW LOG
  }

} else {

  if (epochFlag == true) {  //checks if timepulse interrupt has occured
    millisAtEpoch = millis(); //sets approxime time of timepulse
    epochFlag = false;        //resets for next timepulse
  }

  if ( processGPS() ) {
    logFile.print(logTime); //point at which message parsing has finished
    logFile.print(",");
    logFile.print(pvt.lat); //latitude
    logFile.print(",");
    logFile.print(pvt.lon); //longitude
    logFile.print(",");
    logFile.print(pvt.hMSL); //altitude at sea level
    logFile.print(",");
    logFile.print(pvt.velN); //velocity north
    logFile.print(",");
    logFile.print(pvt.velE); //velocity east
    logFile.print(",");
    logFile.print(pvt.velD); //velocity down
    logFile.print(",");
    logFile.print(pvt.numSV); //number of satellites
    logFile.print(",");
    logFile.print(pvt.gSpeed); //ground Speed mm/s
    logFile.print(",");
    logFile.print(pvt.sAcc); //speed accuracy estimate
    logFile.println();

    ADCsamples    = 0;                       //reset the ADC samples couter to 0
    lastADCsample = millis() - ADC_INTERVAL; // force the 1st ADC sample

    if ( (millis() - millisAtEpoch) > 200 and (noGpsFlag == false)) {     //asumes no gps fix when no epoch recieved within 200ms.
      noGpsFlag = true;
    }

            if ((ADCsamples < MAX_ADC_SAMPLES) and
                ((millis() - lastADCsample) >= ADC_INTERVAL) or ((noGpsFlag == true)   //when GPS outage, operates indefinitly at set sample rate
                    and ((millis() - lastADCsample) >= ADC_INTERVAL)) )
            {
              
          uint8_t inputValues [10];

          for(int i=0; i<10; i++){
            logTime += ADC_INTERVAL;
          inputValues[i] = analogRead(A5);
//               logFile.print(logTime);
//              logFile.print(',');   //  <-- needed?
          logFile.println(inputValues[i]);
          
          }              

          ADCsamples++;
          lastADCsample += ADC_INTERVAL;
  }

}
  }
}
}
//------------------------------------------------------------------------------
// FLAG SWICTH WHEN INTERRUPS COMES FROM PPS OF GPS

void isr() {
  epochFlag = true;   //timepulse has indicated epoch
  noGpsFlag = false;  //false as GPS is recieving messages
}


bool processGPS() {                         //loop comes here to check if a valid message is available.
  static int fpos = 0;
  static unsigned char checksum[2];
  const int payloadSize = sizeof(NAV_PVT);

  while ( Serial1.available() ) {
byte c = Serial1.read();
if ( fpos < 2 ) {
  if ( c == UBX_HEADER[fpos] )
    fpos++;
  else
    fpos = 0;
}
else {
  if ( (fpos - 2) < payloadSize )
    ((unsigned char*)(&pvt))[fpos - 2] = c;

  fpos++;

  if ( fpos == (payloadSize + 2) ) {
    calcChecksum(checksum);
  }
  else if ( fpos == (payloadSize + 3) ) {
    if ( c != checksum[0] )
      fpos = 0;
  }
  else if ( fpos == (payloadSize + 4) ) {
    fpos = 0;
    if ( c == checksum[1] ) {
      return true;
    }
  }
  else if ( fpos > (payloadSize + 4) ) {
    fpos = 0;
  }
}
  }
  return false;
}

void calcChecksum(unsigned char* CK) {            //Checks if message has been parsed correctly
  memset(CK, 0, 2);
  for (int i = 0; i < (int)sizeof(NAV_PVT); i++) {
CK[0] += ((unsigned char*)(&pvt))[i];
CK[1] += CK[0];
  }
}

The loop to log the analog signal written like this is simply not working, but i do not know how to write it in a good way.

#8

[quote="alek91, post:7, topic:703910"]
Probably the creation of the file can be simplied
[/quote]Very probably true. Since you are using time somewhere, you might try using time as a filename instead of all that other stuff. The code is mostly meaningless to me, as I know nothing about GPS, and have never heard of usertypes.lib, but there IS a list of prints to SD which looks normal. There is probably more than one way of doing what I'm guessing you want. One is to have three lists with count conditions on which is used. This is so that they can have " " blanks in the places where sensors are not read, thereby allowing you to have all the columns line up. You have not made any attempt to separate the reading according to the frequencies you say you want, and indeed there is no timing in the loop at all, which may may well be part of your problem. I imagine the loop for this need be nothing but the 100Hz job, the counts and flags for the other jobs, all in a 100Hz window.

#9

I don't like the idea to have blanks in the file . Is possible with SdFat to create different files logged at the same time. at that point i will create one file for gps and one for the sensors at 100Hz

#10

You can have several files running but you need to ensure that you can have more than one file open at the same time, which may be your justification for using SDFat. Then you need to make up your own mind about your final objective and how you get there. Constantly opening and closing files can eventually lead to grief, hence my previous.

#11

Since in my code the file is opened just once at the beginning and closed at the end whe swithc is turned off, do you think is possible open and close multiple file together ?

#12

No.
When I said

it would have been better to say
"your programme may access several files........ ladedah"
I say no only in the light of the standard SD library. I have no comment, only a caution, on the SDFat library.

#13

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.