SDfat building buffer

Hi,

First I should state that I am using a Teensy 3, however it seems that the SDfat library is the same for both, so I thought I could also ask this question here. I am building a program that is spitting out serial that I am then reading into labview, however I also want to record data straight to an SD card for a back up, plus in the future I hope to implement a stand alone logger. So I have been trying to use the SDfat library to log to an SD card by building a buffer, however I have events that are happening 60 times per revolution. A couple of those events only change once per RPM and only need to record them once. I also have an event that happens 60 times, this is the time between each tooth of a sprocket. Within each sprocket I am collection data that happens between 10-20 times (a hz reading that varies). Below is a sample readout of my serial data.

I1D37926C6546P354H1303,1303,1303,1305,1307,1307,1308,1308,1310,1308,1310,1312,1312,1314,1314,1314,1315,1317,1317,1319
I1D43221C6546P360H1379,1383,1379,1381,1383,1381,1386,1385,1385,1385,1385,1386,1388,1388,1388,1390,1392,1390,1390,1394
I2D45959C1780P6H502,502,502,502,502,502,512,512,512,512,512,512,512,512,512,512,512,512,512,512
I2D53068C1780P12H558,559,560,563,565,565,568,570,572,573,574,576,579,580,582,583,586,587,588,590
I2D63532C1780P18H663,665,666,668,670,672,672,674,676,677,678,680,682,683,684,686,687,689,690,692
I2D78887C1780P24H791,792,794,795,796,798,800,800,803,802,803,805,807,807,809,811,811,811,814,815
I2D104796C1780P30H950,951,949,951,953,955,955,956,957,959,959,959,962,965,963,965,965,965,968,968

Sorry if difficult to read, in my program is finds the letter and then spits out any numbers after it, so first it looks for ‘I’ and spits out 1, then looks for ‘D’ with is time between sprocket , so 37926 in the first line.
So I want to build a buffer of everything after the ‘H’, so up to 1200 values per RPM, a buffer of Cadence time (60 per RPM), and then just one value for ‘I’, and ‘C’. I have been trying to do this, however I can’t seem to build a buffer of HZ values, it just resets. Below is my code that I thought would work since I flush the buffer once per revolution, and assumed the HZ buffer would just build until I flush it. I then write it to the SD card. I am new to coding, so learning as I go, and have spent many hours trying to get this to work and have run out of ideas. Maybe there is a better option out there, however it seems SDfat is the choice when needing to write data to a card at a fast rate. I can post complete code if needed. I have also tried to write per degree, however it misses a lot of data. So I thought building a buffer and writing once per revolution would be better. Also this is for a cycling project, so RPM is between60-120, so I would have between 1s to 0.5s to print to card per revolution. Thanks in advance.

  obufstream bout(buf, sizeof(buf));
  if(bUpdateFlags & DEGREE_FLAG ){
    Serial.print ("I");
    Serial.print (IndexIn);
    Serial.print ("D");
    Serial.print (DegreeIn);
    Serial.print ("C");
    Serial.print (CadenceInShared);
    bout <<  DegreeIn;
    Serial.print ("P");
    Serial.print (PositionIn);
    Serial.print ("H");
    for (i=0; i<BufferIn; i++) { //print HZ array
      Serial.print(HZIn[i]); 
      bout << ',' << HZIn[i];
      if (i<BufferIn-1){
        Serial.print(",");
      }
    }

   
    // cout << buf;
    Serial.println();
    
  } // END Serial print per degree


  if(bUpdateFlags & INDEX_FLAG){
    //obufstream bout (buf, sizeof(buf));
    bout <<  IndexIn << ',' << CadenceInShared << endl;
    //cout << buf ;
    //log data and flush to SD
    logfile << buf << flush; 
  }

http://snippets-r-us.com is up the road away. They specialize in dealing with snippets. Here, we need to see ALL of your code.

What Arduino are you using? The ones with the 328 chips have 2K or SRAM. The SD class uses a buffer to speed up writes to the card. Most of the time, it only writes to the buffer. When the buffer is (almost) full, it writes the buffer to the card then writes the data to the buffer. That buffer is 512 bytes - 1/4 of your memory.

So I want to build a buffer of everything after the 'H', so up to 1200 values per RPM

1200 values per revolution per minute? Or 1200 values per revolution?

If you have a 328 based Arduino, and those values are more than one byte in size, you do not have enough memory.

I am using a Teensy 3, so have 16k of memory, just posted here since a lot of ppl use SDfat on their arduino. From my testing the most values I will record per revolution is 1200, so that means I will have to write that to the SD card every 0.5-1s (all depends on how quickly the person is cycling). In my first post I show sample of what the output is like, each line represents a set of data from every 6 degree of revolution.I will post my complete code so you can make more sense of it. I just assumed that the buffer would fill until I flush it after writing the card, which I call each time one revolution is complete, however when i tried this I don’t get any values, just the values I call in the If statement where I write to the card. I have worked on a second version that just builds an array per revolution and then writes that, however all of my equipment is my lab at school and have not had a chance to try it.

#include <SdFat.h>
#define ECHO_TO_SERIAL   1  // echo data to serial port if nonzero

#define SD_CHIP_SELECT  10  // SD chip select pin
#define hzPin 5
#define degPin 3
#define cadencePin 4
#define buttonpin 2 //push button to turn off logging
#define DEGREE_FLAG 1
#define INDEX_FLAG 2

const int BUFFERSIZE=20;
// shared variables are updated by the ISR and read by loop.
volatile uint8_t bUpdateFlagsShared; //hold the updated flag
volatile int DegreeInShared;
volatile int CadenceInShared;
volatile uint8_t BufferInShared = 0;
volatile int HZInShared[BUFFERSIZE];
volatile int IndexInShared;
volatile int PositionInShared;

unsigned long HZStart;
unsigned long HZPeriod;
unsigned long HZ;
unsigned long DegreeStart;
unsigned long CadenceStart;
unsigned long lastHZ=0;
unsigned long transientTime = 0;
unsigned long elapsedTime=0;
unsigned long pulseWidth;
unsigned long pulseWidthLow;
boolean level=0;
boolean lastLevel=0;
uint8_t i=0;
int offset = 0;
int ch;
int buttonState = 0;

//SDfat variables
char buf[1250];
#define error(s) sd.errorHalt_P(PSTR(s))

// file system object
SdFat sd;

// text file for logging
ofstream logfile;

// Serial print stream
ArduinoOutStream cout(Serial);

void setup(void) 
{ 
  Serial.begin(115200); 
  pinMode(hzPin, INPUT); 
  pinMode(degPin, INPUT); 
  pinMode(cadencePin, INPUT); 
  pinMode(buttonpin, INPUT);
  attachInterrupt(hzPin, counterhz, CHANGE);
  attachInterrupt(degPin, counterdeg, RISING);
  attachInterrupt(cadencePin, doIndex, RISING);

  // initialize the SD card at SPI_HALF_SPEED to avoid bus errors with
  if (!sd.begin(SD_CHIP_SELECT, SPI_HALF_SPEED)) sd.initErrorHalt();

  // create a new file in root, the current working directory
  char name[] = "LOGGER00.CSV";

  for (uint8_t i = 0; i < 100; i++) {
    name[6] = i/10 + '0';
    name[7] = i%10 + '0';
    if (sd.exists(name)) continue;
    logfile.open(name);
    break;
  }
  if (!logfile.is_open()) error("file.open");

  cout << pstr("Logging to: ") << name << endl;
  cout << pstr("Type any character to stop\n\n");

  // format header in buffer
  obufstream bout(buf, sizeof(buf));

  bout << pstr("Index,Degree,Cadence,HZ");
  // for (uint8_t h = 0; h < BUFFERSIZE; h++) {
  // bout << pstr(",HZ") << int(h);
  // }

  logfile << buf << endl;
}

void loop(void) 
{
  static int DegreeIn;
  static int CadenceIn;
  static int HZIn[BUFFERSIZE];
  static int IndexIn; //Do I need to do this, will it reset to 0 each time?
  static int PositionIn;
  static uint8_t bUpdateFlags;
  static uint8_t BufferIn = 0;

  if (Serial.available()>0){ //reset index to 0 for labview
    Serial.read();
    IndexInShared = 0;
    offset=0;
    Serial.flush();
  }

  noInterrupts(); // turn interrupts off quickly while we take local copies of the shared variables
  if(bUpdateFlagsShared)
  {

    // take a local copy of which channels were updated in case we need to use this in the rest of loop
    bUpdateFlags = bUpdateFlagsShared;

    if(bUpdateFlags & DEGREE_FLAG)
    {
      DegreeIn = DegreeInShared;
      BufferIn = BufferInShared;
      IndexIn = IndexInShared;
      for (int k=0; k<BufferIn; k++)
      {
        HZIn [k] = HZInShared [k]; //copy HZ array
      }
      PositionIn = PositionInShared;
    }
    bUpdateFlagsShared = 0;
    BufferInShared=0; 
  }

  interrupts(); 
  obufstream bout(buf, sizeof(buf));
  if(bUpdateFlags & DEGREE_FLAG ){
    Serial.print ("I");
    Serial.print (IndexIn);
    Serial.print ("D");
    Serial.print (DegreeIn);
    bout <<  DegreeIn;
    Serial.print ("C");
    Serial.print (CadenceInShared);
    Serial.print ("P");
    Serial.print (PositionIn);
    Serial.print ("H");
    for (i=0; i<BufferIn; i++) { //print HZ array
      Serial.print(HZIn[i]); 
      bout << ',' << HZIn[i];
      if (i<BufferIn-1){
        Serial.print(",");
      }
    }
    Serial.println(); 
  } // END Serial print per degree


  if(bUpdateFlags & INDEX_FLAG){
    //obufstream bout (buf, sizeof(buf));
    bout <<  IndexIn << ',' << CadenceInShared << endl;
    //cout << buf ;
    //log data and flush to SD
    logfile << buf << flush; 
  }	

  bUpdateFlags = 0;
  if (!Serial.available()) return;//might have to change so logfile does not close premature
  logfile.close(); //add pushbutton to stop logging? logfile.sync()
  cout << pstr("Done!");
}

void doIndex() { //Cadence
  PositionInShared=0;  //reset to Position 0
  CadenceInShared=millis()-CadenceStart;
  CadenceStart=millis(); 
  IndexInShared++;
  bUpdateFlagsShared |= INDEX_FLAG;
}

void counterhz() //Frequency
{ 

  HZStart=micros();
  elapsedTime = HZStart - transientTime;
  level = digitalRead(hzPin);
  if((level != lastLevel) & (elapsedTime > 20)) // if transient, 100us is jitter, prevent false HZ
  { 
    lastLevel = level;
    transientTime = HZStart;   
    if(level == HIGH) pulseWidthLow = elapsedTime;
    else pulseWidth = elapsedTime; 
  }
  HZPeriod = pulseWidth + pulseWidthLow;
  //HZPeriod=micros()- HZStart;
  HZ=1000000 / HZPeriod;
  //&& pulseWidthLow<50 add for SRM
  //Will have to figure out max HZ  
  if ( BufferInShared < BUFFERSIZE && HZ >430 && HZ <2500 ) { //fill array with all HZ per chaing ring tooth
    HZInShared[BufferInShared++] = HZ;
  }
}

void counterdeg()// Degree Time
{ 
  DegreeInShared=micros()-DegreeStart;
  DegreeStart=micros(); 
  PositionInShared+=6;
  bUpdateFlagsShared |= DEGREE_FLAG;

}