Aquisition rate with SD Card

with pleasure.

The setup is a standard setup for my different sketches. The output simply outputs a few standard values ​​/ tests. Some arduinos use software serial. Software serial does not have a buffer. When you write data to this soft serial port, the arduino blocks for a short time (until the data was written). Therefore it is sometimes interesting to check whether and how large the buffer of a serial connection is. Has nothing to do with your project.

It’s very good that you’ve been busy with bit shifting. With the amount of data it is sensible to make an efficient memory technology. So you have less problems later.

Here is the way I have converted the data and saved:

      unsigned long sensorTime   = millis();
      unsigned int  sensor1Value = analogRead(PIN_SENSOR_1);
      unsigned int  sensor2Value = analogRead(PIN_SENSOR_2);
      unsigned int  sensor3Value = analogRead(PIN_SENSOR_3);
      byte sensor1Byte1 = sensor1Value;       //saves first 8 bit
      byte sensor1Byte2 = sensor1Value >> 8;//shifts 8 bit to right, left bits will be zero
      byte sensor2Byte1 = sensor2Value;
      byte sensor2Byte2 = sensor2Value >> 8;
      byte sensor3Byte1 = sensor3Value;
      byte sensor3Byte2 = sensor3Value >> 8;
      //fill byte block to write data efficient to sd card
      byte writeBinDataBlock[6]{sensor1Byte1,sensor1Byte2,sensor2Byte1,sensor2Byte2,sensor3Byte1,sensor3Byte2};
      //write data and add the number written bytes to debugFileWriteByteCount
      debugFileWriteByteCount += file.write(writeBinDataBlock, sizeof(writeBinDataBlock));

U asked for the library to use for sd card’s. i have testet several (all from user fat16lib).
There are many libraries that build on each other.
The SD library is almost a wrapper for the other libraries, which are rather low level.
To work quickly without spending much time, most people use the SD library. To work more effectively you should use the SdFat library. To work very effectively, you can work with the SdFatEx library. There are even libraries to work with the the FileAllocationVolume, the FileAllocationCache, …

As you go to the LowLevel library, the more complicated the work will be, but you “can” work very efficiently.
There are also hidden class and functions to read low level sd card data. On this level you have to worry about the fragmentation of files on the sd card. :frowning:
I would recommend using the SdFat library. It is sufficiently fast and you have a lot of control over the work with the sd card. And u can still work with the File class. You do not have to work with SDFile or FatFile.
Please install the SdFat Library with the library manager, search for “sdfat” its written by Bill Greiman!
I have used this as follows:

#include <SPI.h>
#include "SdFat.h"
...
const int PIN_INT_SD_CS = SDCARD_SS_PIN;//the const SDCARD_SS_PIN is only present on Arduino MKRzero with internal sd card reader! Change SDCARD_SS_PIN to ur CS Pin!
...
SdFat sd;
String  fileName = "sensor.dat";
File    file;
...
//Setup:
Serial.println(" - Init SD Card");
while(!sd.begin(PIN_INT_SD_CS)){ Serial.println("   failed"); delay(1000); } Serial.println("   done");//SDfat
//the while loop waits until the sd card is initialized, if u start the arduino without to insert an sd card, its waiting until u put in the sd card, checking every sec.
...
//Loop / when u start recording:
file = sd.open(fileName, O_WRITE | O_TRUNC);//ATTENTION: resets ur file to 0 bytes, take care. A previous measurement can be overwritten!!! User of ur Project could be angry. ;o)

Just a note: Think of the type of storing the bytes. U have stored ur higher bits in the first byte. I have stored the higher bits in the sec. byte. That’s when the people ask for the byte order.

      //Little-Endian
      byte sensor1Byte1 = sensor1Value;
      byte sensor1Byte2 = sensor1Value >> 8;
      //Big-Endian
      byte sensor1Byte1 = sensor1Value >> 8;
      byte sensor1Byte2 = sensor1Value;

At the later convert back into a number this must be taken into account.

I have written a test tool for Win10 to get back the integer sensor values. its very quick and dirty but can read and convert all the data of a 24h measurement in ~20 sec. have tested it with the first 10000 sensor data:

but I get the following in the txt file: http://imgur.com/HCN48Yn Can you help me out?

Sorry, missed this question. I think u have read the file with e.g. Notepad.exe! This is a Text Viewer and only reads text!! Please think about the storage again.

Ur first try was to myFile.println(EMG_value);! This would save ur sensor value as Ascii Text ("1023" as 4 byte)! See Ascii Table Images on Google.

Now we did convert the Integer Value to 2 raw binary byte! Its not text. so u need a tool wich reads every 2 byte and convert it back to the original integer sensor value!

I have done this (see attached image of my prev. post).

Thank you! With your help I have written a test code, in order to get to know the times between aquisitions and their average (code bellow!).
I’m still having some problems in my tests:

Test results:
Test 1: Doing Serial.println(millis()) at the beggining of EACH cycle
a) Without doing Serial.println(sensors) x 3 (only writing to SD card):

  • average time between aquisition: 5.96 ms
  • min time between aquisition: 0 ms
  • max time between aquisition: 8ms
    b) Doing Serial.println(sensors) x 3 and writing to SD card:
  • average time between aquisition: 22.3 ms
  • min time between aquisition: 1 ms (at first cycle)
  • max time between aquisition: 20 ms

Test 2: Doing Serial.println(millis()) at the beggining of the for loop and at the end
a) Without doing Serial.println(sensors) x 3 (only writing to SD card):

  • average time between aquisition: 15.6 ms
    b) Doing Serial.println(sensors) x 3 and writing to SD card:
  • average time between aquisition: 1.053 ms

I also tested with a sketch that made aquisitions for only 1 second and counted how many there were at the end. I got like 2956 or so.

Questions:

  1. In my test 2)b I got positive results. Negative results at test 1)a can be due to printing the millis? If so, how did you do your tests?
  2. Is there a way to print the sensor values and still get a maximum of around 3ms between samples? I don’t know if you included this in your tests :). I want this because I want to be possible to watch the signal at the pc (if connected to it) while still recording to SD card.
  3. How can I be sure that, although of the positive results in test 2)b, I didn’t had an outlier (per example 5 ms).
    Please see post below also!!!

Code

//aquisition variables
const int EMG_pin = A0;  // Analog input pin that the EMG signal is attached to
lunsigned int EMG_value;
unsigned int PZT_value;
unsigned int ACL_value;

//SD Card libraries & variables
#include <SPI.h>
#include "SdFat.h"
SdFat sd;
File myFile;

//SD Card to-save-variavles
byte EMGByte1;
byte EMGByte2;
byte PZTByte1;
byte PZTByte2;
byte ACLByte1;
byte ACLByte2;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);

  Serial.print("Initializing SD card...");

  if (!sd.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  myFile = sd.open("test.dat", O_WRITE | O_TRUNC);
  Serial.println(millis());
  for (int i = 1 ; i <=1000 ; i++) {
    //Serial.println(millis());
    EMG_value = analogRead(EMG_pin);
    PZT_value = analogRead(EMG_pin);
    ACL_value = analogRead(EMG_pin);
    //Serial.println(EMG_value);
    //Serial.println(PZT_value);
    //Serial.println(ACL_value);
    EMGByte1 = EMG_value; //saves first 8 bit
    EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
    PZTByte1 = PZT_value;
    PZTByte2 = PZT_value >> 8;
    ACLByte1 = ACL_value;
    ACLByte2 = ACL_value >> 8;
    byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
    myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
  }
  Serial.println(millis());

  myFile.close();
}

void loop() {
}

I wrote the following code in order to test if, using the method I asked you in the previous post, I got the right acquisition frequency:

    int init_time = micros();
    int i = 0;
    while(micros() - init_time <= 1000000){
    //Serial.println(millis());
    last_aq_time = micros();
    EMG_value = analogRead(EMG_pin);
    PZT_value = analogRead(EMG_pin);
    ACL_value = analogRead(EMG_pin);
    //Serial.println(EMG_value);
    //Serial.println(PZT_value);
    //Serial.println(ACL_value);
    EMGByte1 = EMG_value; //saves first 8 bit
    EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
    PZTByte1 = PZT_value;
    PZTByte2 = PZT_value >> 8;
    ACLByte1 = ACL_value;
    ACLByte2 = ACL_value >> 8;
    byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
    myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
    i++;
    while(micros()-last_aq_time < 3000){}
    }
    Serial.println(i);

With this code, testing for 1 second, I got 333 aquisitions!
BUT testing with 2 seconds I got 665 aqs and for 3 second I got 998.
I have tested for 9 seconds without the variable i, only writing, and the file had 2992 acquisitions (333*9 = 2997).
Without the code to force the aquisitions every 3 ms, I ran the code and got 2222 acquisitons in ONE second.

Also, trying the code above and changing the time of the while loop to 2993, I get 333 acqs at 1 second and 999 at 3 seconds, but, for example at longer times, it fails to lower values than expected :\

And can I assure that all THREE sensors got acquired with a frequency of 333Hz? It makes a bit confusion because there is a little delay between each aquisition of each sensor (?)

EDIT: Trying the following code I got also less than what was expected (30 acquisitions less):

    int i = 0;
    int init_time = micros();
    while(micros() - init_time <= 60000000){
    last_aq_time = micros();
    i++;
    while(micros()-last_aq_time < 3000){}
    }

(See post bellow also)

diogotec:
And can I assure that all THREE sensors got acquired with a frequency of 333Hz? It makes a bit confusion because there is a little delay between each aquisition of each sensor (?)

If you are using delay(), it is a blocking function.
Nothing gets done when it is called.

If you are using delay(), it is a blocking function. Nothing gets done when it is called.

See the code.. I posted it.

@DrDooom, using your method, which you used on a post you sent me, I am now suceeding with the acquistion rate. Thank you once again!
I have tested it with 20 minutes and it works perfectly :slight_smile:

→ I had to had O_CREAT since it didn’t created a new file without it!
This only has a problem: micros() overflow at 70 minutes!

 myFile = sd.open("test.dat", O_CREAT | O_WRITE | O_TRUNC);
  unsigned long i = 0;
  unsigned long init_time = micros();
  while (micros() - init_time <= 3600000000) {
    EMG_value = analogRead(EMG_pin);
    PZT_value = analogRead(EMG_pin);
    ACL_value = analogRead(EMG_pin);
    EMGByte1 = EMG_value; //saves first 8 bit
    EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
    PZTByte1 = PZT_value;
    PZTByte2 = PZT_value >> 8;
    ACLByte1 = ACL_value;
    ACLByte2 = ACL_value >> 8;
    byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
    myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
    i++;
    while (micros() < init_time + i * 3000) {}
  }
  Serial.println(i);
  myFile.close();

Hmm, ok. You did not respond to post # 20 and # 21. But you have integrated the SdFat library. Therefore, I assume that the problem of "storage" is done and the difference between storing text / bytes / little big endian is clear?

At least you have concerned the problem of frequency.

I was not aware that you need exactly 333.333hz. I thought it would be the rough / min frequency. I think you have to mind how you want to build your code. At first you can hold that in my opinion a delay() or delayMicroseconds() within the loop() should be avoided absolutely!!! Your while loop with "do nothing" is like a delay() or delayMicroseconds() and blocks the arduino like ieee488 told! A delay only makes sense in the setup() in certain cases.

Your program will include the following tasks: [high priority] ecquire sensor data [high priority] save sensor data (sd card) [low priority] serial output [low priority] view on display

Maybe you want to control the start / stop with buttons? Anyway. I would recommend you to structure your program so that you can easily implement new requirements. I solve this as you did in your first post.

if(millis() - last_aq_time >= aq_delay){...}

So you make sure you always have the same frequency. But 1.: millis() can already partly be elapsed (2.0, 2.5 or 2.9). Especially at start. Millis() gives only the total millisecond as non floating number. So it makes sense to use the higher accuracy of micros() (as you did last). But 2.: You do more in your code than just the most important / high priority tasks. In addition to requesting the sensor data and storing the sensor data, you still want to output it on a display and pass it to Serial. I think you will not be able to process everything 333 * per second over a longer duration. Especially the approach of displays mostly takes a lot of time! U should separate the tasks. -> see code below.

The Next Problem is, that everything in ur code consumes time. Even the call millis(). So: ** if(millis() - last_aq_time >= aq_delay){** ** last_aq_time = millis();** CAN cause the problem that millis () is different in the first call than in the second call! So you should set to the fixed values ​​of the variable and not to a second call of millis(). eg.: ** if(millis()>=next_aq_time){** ** next_aq_time += aq_delay;//Next ecquire after a fixed time**

Test code to keep a certain frequency and still handle several tasks (in this test i dont read analog data, just simulating some tasks):

const long SERIAL_BAUD  = 9600;//Serial

bool          acquireSensorActive = false;
unsigned long acquireSensorDataNextMicros = 0;
unsigned long acquireSensorDataDelayMicros = 3000;
unsigned long acquireSensorCount = 0;
unsigned long acquireSensorStartMicros = 0;
unsigned long acquireSensorEndMicros = 0;

//another task
unsigned long trySendToSerialNextMicros = 0;
unsigned long trySendToSerialDelayMicros = 5 * 1000;//5 ms
String        trySendToSerialBuffer = "";//Buffer to store some Text

//another task
unsigned long viewOnDisplayNextMicros = 0;
unsigned long viewOnDisplayDelayMicros = 100 * 1000;//100 ms
void setup() {
  Serial.begin(SERIAL_BAUD);
  delay(500);
  while (!Serial);//wait for serial connection
  Serial.println("acquirement started");
  acquireSensorCount = 0;
  acquireSensorActive = true;
  acquireSensorStartMicros = micros();
  acquireSensorEndMicros = acquireSensorStartMicros + (6 * 1000 * 1000);//End is start + n sec
  acquireSensorDataNextMicros = acquireSensorStartMicros;
}
void loop() {
  if(acquireSensorActive){
    if(micros()>=acquireSensorDataNextMicros){//1. Task -> acuire data
      acquireSensorDataNextMicros += acquireSensorDataDelayMicros;
      //Ecquire Sensor Data, nothing else!
      acquireSensorCount++;
    }
    if(micros()>=acquireSensorEndMicros){//2. Task -> check end or not
      acquireSensorEndMicros = micros();
      acquireSensorActive = false;
      float acquireSensorDurationMicros = acquireSensorEndMicros-acquireSensorStartMicros;
      float acquireSensorDurationMs  = acquireSensorDurationMicros / 1000.0;
      float acquireSensorDurationSec = acquireSensorDurationMicros / 1000.0 / 1000.0;
      Serial.println("");
      Serial.println("acquirement finished");
      Serial.println(" - acquireSensorCount:       " + (String)acquireSensorCount + "");
      Serial.println(" - acquireSensorDuration:    " + (String)acquireSensorDurationMicros + " micros (" + (String)(acquireSensorDurationMicros/1000.0) + " ms)");
      Serial.print  (" - acquire per sec:          "); + Serial.println(acquireSensorCount / acquireSensorDurationSec, 5);
    }
    if(micros()>=trySendToSerialNextMicros){//3. Task try to send data to serial
      trySendToSerialNextMicros += trySendToSerialDelayMicros;
      //Sending Data to Serial... Try to consume less than 2000-3000 micros. Otherwise the next loop can not continue with the ecquire of sensory data
      if(trySendToSerialBuffer.length()==0){
        trySendToSerialBuffer = "Sensor Data #" + (String)acquireSensorCount + "...\r\n";
      } else {
        if(Serial.availableForWrite()>0){//Is there a serial hardware buffer? Only send if we do not block the arduino loop and can use the fast serial buffer! Take care, software serials dont use buffer, they block arduino immediately.
          Serial.print(trySendToSerialBuffer[0]);//Every Loop only send 1 char
          trySendToSerialBuffer = trySendToSerialBuffer.substring(1);//consume the char we have send
        }
      }
    }
    if(micros()>=viewOnDisplayNextMicros){//4. Task display data
      viewOnDisplayNextMicros += viewOnDisplayDelayMicros;
      //Display Data... Try to consume less than 2000-3000 micros. Otherwise the next loop can not continue with the ecquire of sensory data
    }
  }
}

Serial Output:

acquirement started
Sensor Data #1...
Sensor Data #1...
Sensor Data #1...
Sensor Data #1...
Sensor Data #2...
Sensor Data #2...
Sensor Data #2...
Sensor Data #20...
Sensor Data #55...
Sensor Data #90...
Sensor Data #125...
Sensor Data #162...
Sensor Data #198...
Sensor Data #235...
Sensor Data #272...
Sensor Data #308...
Sensor Data #345...
Sensor Data #382...
Sensor Data #418...
Sensor Data #455...
Sensor Data #492...
Sensor Data #528...
Sensor Data #565...
Sensor Data #602...
Sensor Data #638...
Sensor Data #675...
Sensor Data #712...
Sensor Data #748...
Sensor Data #785...
Sensor Data #822...
Sensor Data #858...
Sensor Data #895...
Sensor Data #932...
Sensor Data #968...
Sensor Data #1005...
Sensor Data #1043...
Sensor Data #1082...
Sensor Data #1120...
Sensor Data #1158...
Sensor Data #1197...
Sensor Data #1235...
Sensor Data #1273...
Sensor Data #1312...
Sensor Data #1350...
Sensor Data #1388...
Sensor Data #1427...
Sensor Data #1465...
Sensor Data #1503...
Sensor Data #1542...
Sensor Data #1580...
Sensor Data #1618...
Sensor Data #1657...
Sensor Data #1695...
Sensor Data #1733...
Sensor Data #1772...
Sensor Data #1810...
Sensor Data #1848...
Sensor Data #1887...
Sensor Data #1925...
Sensor Data #1963...

acquirement finished
 - acquireSensorCount:       2000
 - acquireSensorDuration:    6000002.00 micros (6000.00 ms)
 - acquire per sec:          333.33322

You see that not all data was transferred to serial. It is only transmitted so quickly without the arduino being blocked. I would also drive the display in this way. There are libraries and other possibilities to handle different tasts. I like to do it this way.

Ur last Question about the overflowing: The two functions millis() and micros() work with the largest possible number "unsigned long". An unsigned long goes back to 0 when it overflows. This also happens with millis after ~50 days and micros after ~70 minutes. When an unsigned long e.g. Is calculated: 4294967280 + 120, the variable contains the value 105. I would therefore always calculate the time for the next execution. This can then overflow but always fits to the overflowed millis() or micros().

There can only be a Problem when next_aq_time = 4294967295; (last micro befor overflow) and something in your code needs some micros and the in the next loop micros has already overflown (micros returns eg 0)!

Really appreciated once again for your help!

You did not respond to post # 20 and # 21. But you have integrated the SdFat library. Therefore, I assume that the problem of “storage” is done and the difference between storing text / bytes / little big endian is clear?

Yeah, this one is more than clear! :slight_smile:

At first you can hold that in my opinion a delay() or delayMicroseconds() within the loop() should be avoided absolutely!!! Your while loop with “do nothing” is like a delay() or delayMicroseconds() and blocks the arduino like ieee488 told!

Yeah, I was thinking this too! it just blocks the arduino yeah!

Maybe you want to control the start / stop with buttons? Anyway. I would recommend you to structure your program so that you can easily implement new requirements.

On point :slight_smile: but only in a final version! But that won’t be a problem.

About the acquisition frequency, I tested 2 ways: the first one base on my previous code (code 1) and then using a 99%-based-on-yours code (code 2).
In both of them I don’t get the expected result, it only has 1 acquisition more!!!
With code 2 SOMETIMES I get the right value, but most of times it’s 1 more…
For 1 second: 334 acqs
For 3 seconds: 1001 acqs
For 6 seconds: 2001 acqs
For 5 minutes ( 300 seconds): 100001 acqs
For 20 minutes (1200 seconds): 400001 acqs

I really don’t understand why, to be honest! In the test I did with the while() causing a delay I always got the right values
Any clue?

Code 1

//in setup():
myFile = sd.open("teste.dat", O_CREAT | O_WRITE | O_TRUNC);
  unsigned long i = 0;
  unsigned long init_time = micros();
  unsigned long acquireSensorDataNextMicros = init_time;
  while (micros() - init_time <= 1200000000) {
    if (micros() >= acquireSensorDataNextMicros) {
      acquireSensorDataNextMicros += 3000;
      EMG_value = analogRead(EMG_pin);
      PZT_value = analogRead(EMG_pin);
      ACL_value = analogRead(EMG_pin);
      EMGByte1 = EMG_value; //saves first 8 bit
      EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
      PZTByte1 = PZT_value;
      PZTByte2 = PZT_value >> 8;
      ACLByte1 = ACL_value;
      ACLByte2 = ACL_value >> 8;
      byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
      myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
      i++;
    }

Code 2:

//aquisition variables
const int EMG_pin = A0;  // Analog input pin that the EMG signal is attached to
unsigned int EMG_value;
unsigned int PZT_value;
unsigned int ACL_value;
unsigned long acquisitions = 0;
unsigned long init_time;
unsigned long acquireSensorDataNextMicros;
unsigned long acquireSensorDataDelayMicros = 3000;
bool          acquireSensorActive = false;

//SD Card libraries & variables
#include <SPI.h>
#include "SdFat.h"
SdFat sd;
File myFile;

//SD Card to-save-variavles
byte EMGByte1;
byte EMGByte2;
byte PZTByte1;
byte PZTByte2;
byte ACLByte1;
byte ACLByte2;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);

  Serial.print("Initializing SD card...");

  if (!sd.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  myFile = sd.open("teste.dat", O_CREAT | O_WRITE | O_TRUNC);
  acquireSensorActive = true;
  acquisitions = 0;
  init_time = micros();
  acquireSensorDataNextMicros = init_time;
}

void loop() {
  if (acquireSensorActive) {
    if (micros() >= acquireSensorDataNextMicros) {
      acquireSensorDataNextMicros += acquireSensorDataDelayMicros;
      EMG_value = analogRead(EMG_pin);
      PZT_value = analogRead(EMG_pin);
      ACL_value = analogRead(EMG_pin);
      EMGByte1 = EMG_value; //saves first 8 bit
      EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
      PZTByte1 = PZT_value;
      PZTByte2 = PZT_value >> 8;
      ACLByte1 = ACL_value;
      ACLByte2 = ACL_value >> 8;
      byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
      myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
      acquisitions++;
    }
    if (micros() - init_time >= 1000000) {
      Serial.println(acquisitions);
      myFile.close();
      acquireSensorActive = false;
    }
  }
}

PS. I have given up of printing the values! not worth :slight_smile:

Well, I try to understand something like this by considering how the variables change with the run time. For the frequency, duration and number of data are important. Since the duration is variable, think about variables with e.g. 9 ms / micros. So: the first acquirement takes place immediately after the start. At the duration 0ms. Than at 3ms, 6ms, 9ms ... at 9ms it stops. So we have exactly 4 acquirements after 9 ms. ((4/9)*1000 = 444,444Hz) Without the first one we would have 3 in 9ms ((3/9)*1000 = 333,333Hz) Use Excel or another tool to calculate the values ​​for 100ms, 1000ms, ....

From my point of view, it makes no sense to carry out the acquirement directly after the start. And a freq. cant be calculated with 1x and duration of 0ms. ((1/0)*1000 = ?) So maybe try to Change in Setup:

AcquireSensorDataNextMicros = init_time + acquireSensorDataDelayMicros;

It is difficult to reach an exact frequency of 333.333hz when the time is variable. The duration can be 999ms, 1000ms or 1001ms without changing the number of measurements. Or you make sure that the measurement is only finished in 3ms intervall.

For what I know of working with bio-data, the first acquisition is at t=0! But you are more than right! I have checked with data that I had on my pc and at t=1 second (and starting at t=0s) we have the frequency acquisitions + 1 :) It's all correct, thanks @DrDooom!

Ur last Question about the overflowing: The two functions millis() and micros() work with the largest possible number "unsigned long". An unsigned long goes back to 0 when it overflows. This also happens with millis after ~50 days and micros after ~70 minutes. When an unsigned long e.g. Is calculated: 4294967280 + 120, the variable contains the value 105. I would therefore always calculate the time for the next execution. This can then overflow but always fits to the overflowed millis() or micros().

There can only be a Problem when next_aq_time = 4294967295; (last micro befor overflow) and something in your code needs some micros and the in the next loop micros has already overflown (micros returns eg 0)!

About the overflow of the micros() I can't actually predict or imagine when I will have a problem! Even if I have next_aq_time = 4294967295, won't micros() be always less than it until it reaches the cycle again and then next_aq_time overflows and goes back to 0 but also does micros()? EDIT: Just ran the code for 80 minutes and I got 1600000. Was expecting 1600001, maybe I lost one during the micros() overflow? OR could be because I used millis() to count the stop time and it has less precision :) Will test now for 2 overflows and see if I lose one more acquisition.

There are also hidden class and functions to read low level sd card data. On this level you have to worry about the fragmentation of files on the sd card. :-(

I've forgotten to ask about this. What do you mean?

For what I know of working with bio-data, the first acquisition is at t=0!

Well, u can change the code / order of events to handle it exactly as desired with first acqusition at t=0 and a working frequency calculation… I would change the code to run allways 3ms interval:

const long SERIAL_BAUD  = 9600;//Serial

bool          acquireSensorRequestStop = false;
bool          acquireSensorActive = false;
unsigned long acquireSensorDataNextMicros = 0;
unsigned long acquireSensorDataDelayMicros = 3000;
unsigned long acquireSensorCount = 0;
unsigned long acquireSensorStartMicros = 0;
unsigned long acquireSensorEndMicros = 0;

void setup() {
  Serial.begin(SERIAL_BAUD);
  delay(500);
  while (!Serial);//Warten bis eine Serielle Verbindung kommt!
  Serial.println("acquirement started");
  acquireSensorRequestStop = false;
  acquireSensorActive = true;
  acquireSensorDataNextMicros = micros();
  acquireSensorStartMicros = micros();
}

void loop() {
  if(acquireSensorActive){
    if(micros()>=acquireSensorDataNextMicros){
      if(acquireSensorRequestStop){
        acquireSensorEndMicros = micros();
        acquireSensorRequestStop = false;
        acquireSensorActive = false;
        
        Serial.println("acquirement stopped");
        Serial.println("acquireSensor Duration:  " + (String)(acquireSensorEndMicros-acquireSensorStartMicros) + " micros (" + (String)((acquireSensorEndMicros-acquireSensorStartMicros)/1000.0) + " ms) -> should fit to 3 ms intervall");
        Serial.println("acquireSensor Count:     " + (String)acquireSensorCount + " x");
        Serial.print  ("acquireSensor Frequency: ");Serial.print((acquireSensorCount / (float)(acquireSensorEndMicros-acquireSensorStartMicros)) * 1000.0 * 1000.0, 3); Serial.println(" Hz");
      } else {
        acquireSensorDataNextMicros += acquireSensorDataDelayMicros;
        //Ecquire Sensor Data...
        acquireSensorCount++;
      }
    } else if(micros()-acquireSensorStartMicros > 1000 * 1000 * 5){ acquireSensorRequestStop = true; }//Dont stop immediately, only request to stop!
  }
}

The last line with acquireSensorRequestStop = true; is important. It does not stop the acquirement, it only requests the stop. The 3ms interval checks if the stop is requested (if(acquireSensorRequestStop){) and stops or acquires more sensor data.

With different durations / stop requests i got this results:

request to stop after 1 ms
acquirement started
acquirement stopped
acquireSensor Duration:  3005 micros (3.00 ms) -> should fit to 3 ms intervall
acquireSensor Count:     1 x
acquireSensor Frequency: 332.779 Hz

request to stop after 10 ms
acquirement started
acquirement stopped
acquireSensor Duration:  12001 micros (12.00 ms) -> should fit to 3 ms intervall
acquireSensor Count:     4 x
acquireSensor Frequency: 333.306 Hz

request to stop after 1000 ms
acquirement started
acquirement stopped
acquireSensor Duration:  1002001 micros (1002.00 ms) -> should fit to 3 ms intervall
acquireSensor Count:     334 x
acquireSensor Frequency: 333.333 Hz

request to stop after 5000 ms
acquirement started
acquirement stopped
acquireSensor Duration:  5001002 micros (5001.00 ms) -> should fit to 3 ms intervall
acquireSensor Count:     1667 x
acquireSensor Frequency: 333.333 Hz

request to stop after 60000 ms
acquirement started
acquirement stopped
acquireSensor Duration:  60003004 micros (60003.00 ms) -> should fit to 3 ms intervall
acquireSensor Count:     20001 x
acquireSensor Frequency: 333.333 Hz

Well, with low durations like < 1000 ms it can be difficult to achieve exactly 333,333 hz.

Maybe u should use an Arduino Due with 84Mhz. I have one, but only tested the code on Arduino MkrZero with 48Mhz.

About the overflow of the micros() I can’t actually predict or imagine when I will have a problem!

I think this is the biggest problem. If the measurements with some users simply do not continue after 70/140/210 … minutes!
The Problem can happen, if…
next_aq_time = 4294967295;
and
micros() is 4294967294 (~1 less than next_aq_time)
if the Arduino is doint something longer than 2 micros and is blocked, the next micros() will overflow to 0.
So
if(millis()>=next_aq_time){
will not be executed and next_aq_time never overflows because there is no next_aq_time += 3000;.

I’ve forgotten to ask about this. What do you mean?

I would not work with the hidden functions. They can be removed in updates or no longer work as before.

Thanks for the code :slight_smile: that issue is also solved now!

About the overflowing: Yeah, I see! Is there any way we can put safeguards?

What would be your approach?

Something like

if((micros()>=acquireSensorDataNextMicros) OR (next_aq_time == 4294967295 AND micros() <=1))

Hmm, I think it could still come to problems. Perhaps you could try that out?

if(micros()>=acquireSensorDataNextMicros || micros() < acquireSensorDataNextMicros - acquireSensorDataDelayMicros){//time >= next run or time is unexpectedly to small

How can I test such simulation? It is unlikely to get into that situation, so test it is hard!

The recommend way to test for an expired interval is like this:

if (millis() - lastMillis >= interval) {
    lastMillis += interval;
    // your code here
}

(That can easily be changed to use micros().)

Just keep track of the last time your event expired, and then subtract it from the current time and see if that's reached your interval. This technique is immune to millis()/micros() wrap-around.

Hey @christop, That way was similar to my first attempt. For some reason I always get 1 or 2 less acquisitions than expected. By the way, for working at overflow shouln't I do abs(millis() - lastMillis)?

Didn't understood tho I can use that code to test the overflow case!

diogotec: Hey @christop, That way was similar to my first attempt. For some reason I always get 1 or 2 less acquisitions than expected. By the way, for working at overflow shouln't I do abs(millis() - lastMillis)?

Didn't understood tho I can use that code to test the overflow case!

Nope, millis() - lastMillis always gives an unsigned long value (both millis() and lastMillis must be unsigned long), so it works out correctly even in the presence of overflow/wraparound.

One way to force it to "expire" immediately (to take an acquisition right away) is to initialize lastMillis to millis() - interval. I'm not sure if that would fix the exact problem you're seeing, though.