Go Down

Topic: Aquisition rate with SD Card (Read 2677 times) previous topic - next topic

diogotec

#15
Jul 21, 2017, 01:55 pm Last Edit: Jul 21, 2017, 02:03 pm by diogotec
Quote
If you want to save data permanently and over a long period, the buffer will make no sense.
Why not?

Quote
The ~ 3ms could be enough to write over more than one block. Maybe you do not even need a continuous writing process. Did you test the data over a longer period of time?
I'm afraid I did not understand what you meant!

Quote
By the way: how high is the resolution of your analog inputs?
I'm using arduino UNO, so 10 bits resolution (0-1023).

The sensors are eletromyography and a piezoeletric sensor (both I don't expect to reach 4 digits number), and an accelerometer. With the accelerometer I only want to save if the person is up (1) or layed down (0).

DrDooom

Quote
If you want to save data permanently and over a long period, the buffer will make no sense.
Quote
Why not?
Well, with a buffer u are collecting data and than u write once every 500 ms. but u have to write e.g. 1000 bytes. this can take longer than ur 3ms (300 Hz). the writing to sd card clearly blocks ur arduino (the loop waits for the sd card writing) and you can't continue with the next sensor aequirement.

so it is better to write after every sensor aequiring. than u need to write e.g. 10 bytes and this should be possible in 3ms. so the arduino will resume with the next loop / and aequire after 3 ms.

diogotec

What you are basically saying is that to use the writing to the SDcard (print + flush?) in order to make the 3ms delay?
Using the SDfat library and the SdFile Class? Can you help me how to use this library?

DrDooom

I have written a test tool to acquire 3 analog values ​​and check the performance for reading / saving data (test result at the end). Hmmm, I do not know why, but the writing performance looks like there are no big latencies like reading over blocks. Maybe I was wrong when I said that there are problems with writing to several blocks.

Before you deal with the writing, however, you should consider how you most effectively save your sensor data.

I think you would most likely save the integer values ​​ascii text?
Code: [Select]

800,920,125
150,193,740
102,591,690
80,1015,1002

But you would have to save for the 3 values ​​(0-1023) + separators up to 15 bytes.
1023,1023,1023\n (total 15 chars)

It is more effective to save the data as bytes.
Since you have 10 bit values ​​(0-1023), you can not save them as a byte (8 bit, 0-255). You need 2 byte's. There is a good possibility to convert a 16 bit variable into 2x 8 bit variable. I recommend you read something about bit shifting.
So if you can store a value (10 bit) in two byte, you need for 3 sensor values ​​only total 6 byte.
The speed and the overall data size is considerably reduced.

Save as Text:
15 byte * 333 Hz * 60 sec * 60 min * 24 Hours = ~ 412 MB
Save as Bytes:
6 byte * 333 Hz * 60 sec * 60 min * 24 Hours = ~ 165 MB





Here is the output of the test (data saved as 6 byte array):
############################################
### Setup ###
 - Serial
   Buffer: 63 bytes
 - Pin Modes
 - Set Pins High/Low
 - Init SD Card
   done
### Setup finished ###
Starting acquire...
 - open sensor data file / truncate
   done! size 0 byte
Stopped acquire!

duration test:        1148 sec (19.13 minutes)
debugCount:           382915x sensor data acquired
debugSensorMin:       1270 micros (1.27 ms) -> min duration to acquire sensor data
debugSensorMax:       1274 micros (1.27 ms) -> max duration to acquire sensor data
debugSensorTotal:     486552494 micros (486552.49 ms) -> total duration to acquire sensor data
debugFileWriteMin:    8 micros (0.01 ms) -> min duration to write the 6 byte data to sd card
debugFileWriteMax:    9 micros (0.01 ms) -> max duration to write the 6 byte data to sd card
debugFileWriteTotal:  8151947 micros (8151.95 ms) -> total duration to write the 6 byte data to sd card
debugFileWriteByteCount: 2297490 bytes (2243.64 kb) -> total bytes written zu sd card
avg   sensor read:      1270.65 micros (1.27 ms)
avg   file write:       21.29 micros (0.02 ms)
perf. file write:       275.23 kb/sec
############################################

Total time 1148 sec and 382915 sensor acquire: the frequency is ~333,54 Hz over 19 Minutes.

The file size on sd card with raw byte data:
Code: [Select]

 Datenträger in Laufwerk F: ist 2GB SD CARD
 Volumeseriennummer: CC86-F45B

01.01.2000  01:00         2.297.490 sensor.dat
               1 Datei(en),      2.297.490 Bytes
               0 Verzeichnis(se),  1.979.711.488 Bytes frei


So saving without a buffer should be no problem!

diogotec

WOW! DrDooom, I am very grateful for you detailed help!
Although I still have some pending questions!

1)
I have read and learned about bit-shifting. I used the following code to obtain the 2 bytes out of my sensor value:

Code: [Select]

byte EMG_byte[2]; //array containing the EMG sensor reading
EMG_byte[0] = (byte) ((EMG_value >> 8) & 0xFF);
EMG_byte[1] = (byte) (EMG_value & 0xFF);


I have tested it using Serial.print, which outputs the bytes in decimal, which I then convert to binary, merge, and convert to decimal, and it works.

Tho, my problem is that I don't know how to print these values to the SD.card. I have tried the following:
Code: [Select]

myFile.write(EMG_byte,2);

but I get the following in the txt file: http://imgur.com/HCN48Yn
Can you help me out?

2) In your report you have:
Quote
Here is the output of the test (data saved as 6 byte array):
############################################
### Setup ###
 - Serial
   Buffer: 63 bytes
 - Pin Modes
 - Set Pins High/Low
 - Init SD Card
   done
### Setup finished ###
Starting acquire...
 - open sensor data file / truncate
   done! size 0 byte
Stopped acquire!
In the setup why do you talk about a buffer of 63 bytes?
Also, in the acquiring section, I am supposed to:
1 - open the file ONCE
2 - write data to it (and flush???)
3 - at the end, 24hours later or so, close the file?


3) In your simulation you used what library?

Thank you very much!!

DrDooom

with pleasure.

The setup is a standard setup for my different sketches. The output simply outputs a few standard values ​​/ tests. Some arduinos use software serial. Software serial does not have a buffer. When you write data to this soft serial port, the arduino blocks for a short time (until the data was written). Therefore it is sometimes interesting to check whether and how large the buffer of a serial connection is. Has nothing to do with your project.

It's very good that you've been busy with bit shifting. With the amount of data it is sensible to make an efficient memory technology. So you have less problems later.

Here is the way I have converted the data and saved:
Code: [Select]

      unsigned long sensorTime   = millis();
      unsigned int  sensor1Value = analogRead(PIN_SENSOR_1);
      unsigned int  sensor2Value = analogRead(PIN_SENSOR_2);
      unsigned int  sensor3Value = analogRead(PIN_SENSOR_3);
      byte sensor1Byte1 = sensor1Value;       //saves first 8 bit
      byte sensor1Byte2 = sensor1Value >> 8;//shifts 8 bit to right, left bits will be zero
      byte sensor2Byte1 = sensor2Value;
      byte sensor2Byte2 = sensor2Value >> 8;
      byte sensor3Byte1 = sensor3Value;
      byte sensor3Byte2 = sensor3Value >> 8;
      //fill byte block to write data efficient to sd card
      byte writeBinDataBlock[6]{sensor1Byte1,sensor1Byte2,sensor2Byte1,sensor2Byte2,sensor3Byte1,sensor3Byte2};
      //write data and add the number written bytes to debugFileWriteByteCount
      debugFileWriteByteCount += file.write(writeBinDataBlock, sizeof(writeBinDataBlock));


U asked for the library to use for sd card's. i have testet several (all from user fat16lib).
There are many libraries that build on each other.
The SD library is almost a wrapper for the other libraries, which are rather low level.
To work quickly without spending much time, most people use the SD library. To work more effectively you should use the SdFat library. To work very effectively, you can work with the SdFatEx library. There are even libraries to work with the the FileAllocationVolume, the FileAllocationCache, ....

As you go to the LowLevel library, the more complicated the work will be, but you "can" work very efficiently.
There are also hidden class and functions to read low level sd card data. On this level you have to worry about the fragmentation of files on the sd card. :-(
I would recommend using the SdFat library. It is sufficiently fast and you have a lot of control over the work with the sd card. And u can still work with the File class. You do not have to work with SDFile or FatFile.
Please install the SdFat Library with the library manager, search for "sdfat" its written by Bill Greiman!
I have used this as follows:
Code: [Select]

#include <SPI.h>
#include "SdFat.h"
...
const int PIN_INT_SD_CS = SDCARD_SS_PIN;//the const SDCARD_SS_PIN is only present on Arduino MKRzero with internal sd card reader! Change SDCARD_SS_PIN to ur CS Pin!
...
SdFat sd;
String  fileName = "sensor.dat";
File    file;
...
//Setup:
Serial.println(" - Init SD Card");
while(!sd.begin(PIN_INT_SD_CS)){ Serial.println("   failed"); delay(1000); } Serial.println("   done");//SDfat
//the while loop waits until the sd card is initialized, if u start the arduino without to insert an sd card, its waiting until u put in the sd card, checking every sec.
...
//Loop / when u start recording:
file = sd.open(fileName, O_WRITE | O_TRUNC);//ATTENTION: resets ur file to 0 bytes, take care. A previous measurement can be overwritten!!! User of ur Project could be angry. ;o)




Just a note: Think of the type of storing the bytes. U have stored ur higher bits in the first byte. I have stored the higher bits in the sec. byte. That's when the people ask for the byte order.
Code: [Select]

      //Little-Endian
      byte sensor1Byte1 = sensor1Value;
      byte sensor1Byte2 = sensor1Value >> 8;
      //Big-Endian
      byte sensor1Byte1 = sensor1Value >> 8;
      byte sensor1Byte2 = sensor1Value;

At the later convert back into a number this must be taken into account.

I have written a test tool for Win10 to get back the integer sensor values. its very quick and dirty but can read and convert all the data of a 24h measurement in ~20 sec. have tested it with the first 10000 sensor data:

DrDooom

#21
Jul 22, 2017, 12:24 pm Last Edit: Jul 22, 2017, 12:29 pm by DrDooom
Quote
but I get the following in the txt file: http://imgur.com/HCN48Yn
Can you help me out?
Sorry, missed this question. I think u have read the file with e.g. Notepad.exe! This is a Text Viewer and only reads text!! Please think about the storage again.

Ur first try was to myFile.println(EMG_value);! This would save ur sensor value as Ascii Text ("1023" as 4 byte)! See Ascii Table Images on Google.

Now we did convert the Integer Value to 2 raw binary byte! Its not text. so u need a tool wich reads every 2 byte and convert it back to the original integer sensor value!

I have done this (see attached image of my prev. post).

diogotec

#22
Jul 23, 2017, 12:34 am Last Edit: Jul 23, 2017, 09:53 pm by diogotec
Thank you! With your help I have written a test code, in order to get to know the times between aquisitions and their average (code bellow!).
I'm still having some problems in my tests:

Test results:
Test 1: Doing Serial.println(millis()) at the beggining of EACH cycle
a) Without doing Serial.println(sensors) x 3 (only writing to SD card):
    - average time between aquisition: 5.96 ms
    - min time between aquisition: 0 ms
    - max time between aquisition: 8ms
b) Doing Serial.println(sensors) x 3 and writing to SD card:
    - average time between aquisition: 22.3 ms
    - min time between aquisition: 1 ms (at first cycle)
    - max time between aquisition: 20 ms

Test 2: Doing Serial.println(millis()) at the beggining of the for loop and at the end
a) Without doing Serial.println(sensors) x 3 (only writing to SD card):
    - average time between aquisition: 15.6 ms
b) Doing Serial.println(sensors) x 3 and writing to SD card:
    - average time between aquisition: 1.053 ms

I also tested with a sketch that made aquisitions for only 1 second and counted how many there were at the end. I got like 2956 or so.

Questions:
1) In my test 2)b I got positive results. Negative results at test 1)a can be due to printing the millis? If so, how did you do your tests?
2) Is there a way to print the sensor values and still get a maximum of around 3ms between samples? I don't know if you included this in your tests :). I want this because I want to be possible to watch the signal at the pc (if connected to it) while still recording to SD card.
3) How can I be sure that, although of the positive results in test 2)b, I didn't had an outlier (per example 5 ms).
Please see post below also!!!

Code
Code: [Select]

//aquisition variables
const int EMG_pin = A0;  // Analog input pin that the EMG signal is attached to
lunsigned int EMG_value;
unsigned int PZT_value;
unsigned int ACL_value;

//SD Card libraries & variables
#include <SPI.h>
#include "SdFat.h"
SdFat sd;
File myFile;

//SD Card to-save-variavles
byte EMGByte1;
byte EMGByte2;
byte PZTByte1;
byte PZTByte2;
byte ACLByte1;
byte ACLByte2;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);

  Serial.print("Initializing SD card...");

  if (!sd.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  myFile = sd.open("test.dat", O_WRITE | O_TRUNC);
  Serial.println(millis());
  for (int i = 1 ; i <=1000 ; i++) {
    //Serial.println(millis());
    EMG_value = analogRead(EMG_pin);
    PZT_value = analogRead(EMG_pin);
    ACL_value = analogRead(EMG_pin);
    //Serial.println(EMG_value);
    //Serial.println(PZT_value);
    //Serial.println(ACL_value);
    EMGByte1 = EMG_value; //saves first 8 bit
    EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
    PZTByte1 = PZT_value;
    PZTByte2 = PZT_value >> 8;
    ACLByte1 = ACL_value;
    ACLByte2 = ACL_value >> 8;
    byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
    myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
  }
  Serial.println(millis());

  myFile.close();
}

void loop() {
}

diogotec

#23
Jul 23, 2017, 02:00 pm Last Edit: Jul 24, 2017, 01:40 pm by diogotec
I wrote the following code in order to test if, using the method I asked you in the previous post, I got the right acquisition frequency:

Code: [Select]
    int init_time = micros();
    int i = 0;
    while(micros() - init_time <= 1000000){
    //Serial.println(millis());
    last_aq_time = micros();
    EMG_value = analogRead(EMG_pin);
    PZT_value = analogRead(EMG_pin);
    ACL_value = analogRead(EMG_pin);
    //Serial.println(EMG_value);
    //Serial.println(PZT_value);
    //Serial.println(ACL_value);
    EMGByte1 = EMG_value; //saves first 8 bit
    EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
    PZTByte1 = PZT_value;
    PZTByte2 = PZT_value >> 8;
    ACLByte1 = ACL_value;
    ACLByte2 = ACL_value >> 8;
    byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
    myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
    i++;
    while(micros()-last_aq_time < 3000){}
    }
    Serial.println(i);


With this code, testing for 1 second, I got 333 aquisitions!
BUT testing with 2 seconds I got 665 aqs and for 3 second I got 998.
I have tested for 9 seconds without the variable i, only writing, and the file had 2992 acquisitions (333*9 = 2997).
Without the code to force the aquisitions every 3 ms, I ran the code and got 2222 acquisitons in ONE second.

Also, trying the code above and changing the time of the while loop to 2993, I get 333 acqs at 1 second and 999 at 3 seconds, but, for example at longer times, it fails to lower values than expected :\

And can I assure that all THREE sensors got acquired with a frequency of 333Hz? It makes a bit confusion because there is a little delay between each aquisition of each sensor (?)

EDIT: Trying the following code I got also less than what was expected (30 acquisitions less):
Code: [Select]

    int i = 0;
    int init_time = micros();
    while(micros() - init_time <= 60000000){
    last_aq_time = micros();
    i++;
    while(micros()-last_aq_time < 3000){}
    }


(See post bellow also)

ieee488

And can I assure that all THREE sensors got acquired with a frequency of 333Hz? It makes a bit confusion because there is a little delay between each aquisition of each sensor (?)

If you are using delay(), it is a blocking function.
Nothing gets done when it is called.


diogotec

Quote
If you are using delay(), it is a blocking function.
Nothing gets done when it is called.
See the code.. I posted it.

diogotec

@DrDooom, using your method, which you used on a post you sent me, I am now suceeding with the acquistion rate. Thank you once again!
I have tested it with 20 minutes and it works perfectly :)

-> I had to had O_CREAT since it didn't created a new file without it!
This only has a problem: micros() overflow at 70 minutes!

Code: [Select]

 myFile = sd.open("test.dat", O_CREAT | O_WRITE | O_TRUNC);
  unsigned long i = 0;
  unsigned long init_time = micros();
  while (micros() - init_time <= 3600000000) {
    EMG_value = analogRead(EMG_pin);
    PZT_value = analogRead(EMG_pin);
    ACL_value = analogRead(EMG_pin);
    EMGByte1 = EMG_value; //saves first 8 bit
    EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
    PZTByte1 = PZT_value;
    PZTByte2 = PZT_value >> 8;
    ACLByte1 = ACL_value;
    ACLByte2 = ACL_value >> 8;
    byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
    myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
    i++;
    while (micros() < init_time + i * 3000) {}
  }
  Serial.println(i);
  myFile.close();

DrDooom

#27
Jul 24, 2017, 02:29 pm Last Edit: Jul 24, 2017, 02:42 pm by DrDooom
Hmm, ok.
You did not respond to post # 20 and # 21. But you have integrated the SdFat library. Therefore, I assume that the problem of "storage" is done and the difference between storing text / bytes / little big endian is clear?

At least you have concerned the problem of frequency.

I was not aware that you need exactly 333.333hz. I thought it would be the rough / min frequency. I think you have to mind how you want to build your code.
At first you can hold that in my opinion a delay() or delayMicroseconds() within the loop() should be avoided absolutely!!! Your while loop with "do nothing" is like a delay() or delayMicroseconds() and blocks the arduino like ieee488 told!
A delay only makes sense in the setup() in certain cases.


Your program will include the following tasks:
[high priority] ecquire sensor data
[high priority] save sensor data (sd card)
[low priority]  serial output
[low priority]  view on display

Maybe you want to control the start / stop with buttons? Anyway. I would recommend you to structure your program so that you can easily implement new requirements. I solve this as you did in your first post.

if(millis() - last_aq_time >= aq_delay){...}

So you make sure you always have the same frequency.
But 1.: millis() can already partly be elapsed (2.0, 2.5 or 2.9). Especially at start. Millis() gives only the total millisecond as non floating number. So it makes sense to use the higher accuracy of micros() (as you did last).
But 2.: You do more in your code than just the most important / high priority tasks. In addition to requesting the sensor data and storing the sensor data, you still want to output it on a display and pass it to Serial. I think you will not be able to process everything 333 * per second over a longer duration. Especially the approach of displays mostly takes a lot of time! U should separate the tasks. -> see code below.

The Next Problem is, that everything in ur code consumes time. Even the call millis(). So:
  if(millis() - last_aq_time >= aq_delay){
    last_aq_time = millis();

CAN cause the problem that millis () is different in the first call than in the second call! So you should set to the fixed values ​​of the variable and not to a second call of millis().
eg.:
    if(millis()>=next_aq_time){
      next_aq_time += aq_delay;//Next ecquire after a fixed time



Test code to keep a certain frequency and still handle several tasks (in this test i dont read analog data, just simulating some tasks):
Code: [Select]

const long SERIAL_BAUD  = 9600;//Serial

bool          acquireSensorActive = false;
unsigned long acquireSensorDataNextMicros = 0;
unsigned long acquireSensorDataDelayMicros = 3000;
unsigned long acquireSensorCount = 0;
unsigned long acquireSensorStartMicros = 0;
unsigned long acquireSensorEndMicros = 0;

//another task
unsigned long trySendToSerialNextMicros = 0;
unsigned long trySendToSerialDelayMicros = 5 * 1000;//5 ms
String        trySendToSerialBuffer = "";//Buffer to store some Text

//another task
unsigned long viewOnDisplayNextMicros = 0;
unsigned long viewOnDisplayDelayMicros = 100 * 1000;//100 ms
void setup() {
  Serial.begin(SERIAL_BAUD);
  delay(500);
  while (!Serial);//wait for serial connection
  Serial.println("acquirement started");
  acquireSensorCount = 0;
  acquireSensorActive = true;
  acquireSensorStartMicros = micros();
  acquireSensorEndMicros = acquireSensorStartMicros + (6 * 1000 * 1000);//End is start + n sec
  acquireSensorDataNextMicros = acquireSensorStartMicros;
}
void loop() {
  if(acquireSensorActive){
    if(micros()>=acquireSensorDataNextMicros){//1. Task -> acuire data
      acquireSensorDataNextMicros += acquireSensorDataDelayMicros;
      //Ecquire Sensor Data, nothing else!
      acquireSensorCount++;
    }
    if(micros()>=acquireSensorEndMicros){//2. Task -> check end or not
      acquireSensorEndMicros = micros();
      acquireSensorActive = false;
      float acquireSensorDurationMicros = acquireSensorEndMicros-acquireSensorStartMicros;
      float acquireSensorDurationMs  = acquireSensorDurationMicros / 1000.0;
      float acquireSensorDurationSec = acquireSensorDurationMicros / 1000.0 / 1000.0;
      Serial.println("");
      Serial.println("acquirement finished");
      Serial.println(" - acquireSensorCount:       " + (String)acquireSensorCount + "");
      Serial.println(" - acquireSensorDuration:    " + (String)acquireSensorDurationMicros + " micros (" + (String)(acquireSensorDurationMicros/1000.0) + " ms)");
      Serial.print  (" - acquire per sec:          "); + Serial.println(acquireSensorCount / acquireSensorDurationSec, 5);
    }
    if(micros()>=trySendToSerialNextMicros){//3. Task try to send data to serial
      trySendToSerialNextMicros += trySendToSerialDelayMicros;
      //Sending Data to Serial... Try to consume less than 2000-3000 micros. Otherwise the next loop can not continue with the ecquire of sensory data
      if(trySendToSerialBuffer.length()==0){
        trySendToSerialBuffer = "Sensor Data #" + (String)acquireSensorCount + "...\r\n";
      } else {
        if(Serial.availableForWrite()>0){//Is there a serial hardware buffer? Only send if we do not block the arduino loop and can use the fast serial buffer! Take care, software serials dont use buffer, they block arduino immediately.
          Serial.print(trySendToSerialBuffer[0]);//Every Loop only send 1 char
          trySendToSerialBuffer = trySendToSerialBuffer.substring(1);//consume the char we have send
        }
      }
    }
    if(micros()>=viewOnDisplayNextMicros){//4. Task display data
      viewOnDisplayNextMicros += viewOnDisplayDelayMicros;
      //Display Data... Try to consume less than 2000-3000 micros. Otherwise the next loop can not continue with the ecquire of sensory data
    }
  }
}


Serial Output:
Code: [Select]

acquirement started
Sensor Data #1...
Sensor Data #1...
Sensor Data #1...
Sensor Data #1...
Sensor Data #2...
Sensor Data #2...
Sensor Data #2...
Sensor Data #20...
Sensor Data #55...
Sensor Data #90...
Sensor Data #125...
Sensor Data #162...
Sensor Data #198...
Sensor Data #235...
Sensor Data #272...
Sensor Data #308...
Sensor Data #345...
Sensor Data #382...
Sensor Data #418...
Sensor Data #455...
Sensor Data #492...
Sensor Data #528...
Sensor Data #565...
Sensor Data #602...
Sensor Data #638...
Sensor Data #675...
Sensor Data #712...
Sensor Data #748...
Sensor Data #785...
Sensor Data #822...
Sensor Data #858...
Sensor Data #895...
Sensor Data #932...
Sensor Data #968...
Sensor Data #1005...
Sensor Data #1043...
Sensor Data #1082...
Sensor Data #1120...
Sensor Data #1158...
Sensor Data #1197...
Sensor Data #1235...
Sensor Data #1273...
Sensor Data #1312...
Sensor Data #1350...
Sensor Data #1388...
Sensor Data #1427...
Sensor Data #1465...
Sensor Data #1503...
Sensor Data #1542...
Sensor Data #1580...
Sensor Data #1618...
Sensor Data #1657...
Sensor Data #1695...
Sensor Data #1733...
Sensor Data #1772...
Sensor Data #1810...
Sensor Data #1848...
Sensor Data #1887...
Sensor Data #1925...
Sensor Data #1963...

acquirement finished
 - acquireSensorCount:       2000
 - acquireSensorDuration:    6000002.00 micros (6000.00 ms)
 - acquire per sec:          333.33322


You see that not all data was transferred to serial. It is only transmitted so quickly without the arduino being blocked. I would also drive the display in this way. There are libraries and other possibilities to handle different tasts. I like to do it this way.


Ur last Question about the overflowing:
The two functions millis() and micros() work with the largest possible number "unsigned long". An unsigned long goes back to 0 when it overflows. This also happens with millis after ~50 days and micros after ~70 minutes.
When an unsigned long e.g. Is calculated: 4294967280 + 120, the variable contains the value 105. I would therefore always calculate the time for the next execution. This can then overflow but always fits to the overflowed millis() or micros().

There can only be a Problem when next_aq_time = 4294967295; (last micro befor overflow) and something in your code needs some micros and the in the next loop micros has already overflown (micros returns eg 0)!

diogotec

#28
Jul 24, 2017, 06:39 pm Last Edit: Jul 24, 2017, 08:37 pm by diogotec
Really appreciated once again for your help!

Quote
You did not respond to post # 20 and # 21. But you have integrated the SdFat library. Therefore, I assume that the problem of "storage" is done and the difference between storing text / bytes / little big endian is clear?
Yeah, this one is more than clear!  :)

Quote
At first you can hold that in my opinion a delay() or delayMicroseconds() within the loop() should be avoided absolutely!!! Your while loop with "do nothing" is like a delay() or delayMicroseconds() and blocks the arduino like ieee488 told!
Yeah, I was thinking this too! it just blocks the arduino yeah!

Quote
Maybe you want to control the start / stop with buttons? Anyway. I would recommend you to structure your program so that you can easily implement new requirements.
On point :) but only in a final version! But that won't be a problem.

About the acquisition frequency, I tested 2 ways: the first one base on my previous code (code 1) and then using a 99%-based-on-yours code (code 2).
In both of them I don't get the expected result, it only has 1 acquisition more!!!
With code 2 SOMETIMES I get the right value, but most of times it's 1 more...
For 1 second: 334 acqs
For 3 seconds: 1001 acqs
For 6 seconds: 2001 acqs
For 5 minutes ( 300 seconds): 100001 acqs
For 20 minutes (1200 seconds): 400001 acqs


I really don't understand why, to be honest! In the test I did with the while() causing a delay I always got the right values
Any clue?

Code 1
Code: [Select]

//in setup():
myFile = sd.open("teste.dat", O_CREAT | O_WRITE | O_TRUNC);
  unsigned long i = 0;
  unsigned long init_time = micros();
  unsigned long acquireSensorDataNextMicros = init_time;
  while (micros() - init_time <= 1200000000) {
    if (micros() >= acquireSensorDataNextMicros) {
      acquireSensorDataNextMicros += 3000;
      EMG_value = analogRead(EMG_pin);
      PZT_value = analogRead(EMG_pin);
      ACL_value = analogRead(EMG_pin);
      EMGByte1 = EMG_value; //saves first 8 bit
      EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
      PZTByte1 = PZT_value;
      PZTByte2 = PZT_value >> 8;
      ACLByte1 = ACL_value;
      ACLByte2 = ACL_value >> 8;
      byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
      myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
      i++;
    }


Code 2:
Code: [Select]

//aquisition variables
const int EMG_pin = A0;  // Analog input pin that the EMG signal is attached to
unsigned int EMG_value;
unsigned int PZT_value;
unsigned int ACL_value;
unsigned long acquisitions = 0;
unsigned long init_time;
unsigned long acquireSensorDataNextMicros;
unsigned long acquireSensorDataDelayMicros = 3000;
bool          acquireSensorActive = false;

//SD Card libraries & variables
#include <SPI.h>
#include "SdFat.h"
SdFat sd;
File myFile;

//SD Card to-save-variavles
byte EMGByte1;
byte EMGByte2;
byte PZTByte1;
byte PZTByte2;
byte ACLByte1;
byte ACLByte2;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);

  Serial.print("Initializing SD card...");

  if (!sd.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  myFile = sd.open("teste.dat", O_CREAT | O_WRITE | O_TRUNC);
  acquireSensorActive = true;
  acquisitions = 0;
  init_time = micros();
  acquireSensorDataNextMicros = init_time;
}

void loop() {
  if (acquireSensorActive) {
    if (micros() >= acquireSensorDataNextMicros) {
      acquireSensorDataNextMicros += acquireSensorDataDelayMicros;
      EMG_value = analogRead(EMG_pin);
      PZT_value = analogRead(EMG_pin);
      ACL_value = analogRead(EMG_pin);
      EMGByte1 = EMG_value; //saves first 8 bit
      EMGByte2 = EMG_value >> 8;//shifts 8 bit to right, left bits will be zero
      PZTByte1 = PZT_value;
      PZTByte2 = PZT_value >> 8;
      ACLByte1 = ACL_value;
      ACLByte2 = ACL_value >> 8;
      byte writeBinDataBlock[6] {EMGByte1, EMGByte2, PZTByte1, PZTByte2, ACLByte1, ACLByte2};
      myFile.write(writeBinDataBlock, sizeof(writeBinDataBlock));
      acquisitions++;
    }
    if (micros() - init_time >= 1000000) {
      Serial.println(acquisitions);
      myFile.close();
      acquireSensorActive = false;
    }
  }
}


PS. I have given up of printing the values! not worth :)


DrDooom

Well, I try to understand something like this by considering how the variables change with the run time. For the frequency, duration and number of data are important.
Since the duration is variable, think about variables with e.g. 9 ms / micros.
So: the first acquirement takes place immediately after the start.
At the duration 0ms. Than at 3ms, 6ms, 9ms ... at 9ms it stops.
So we have exactly 4 acquirements after 9 ms. ((4/9)*1000 = 444,444Hz)
Without the first one we would have 3 in 9ms ((3/9)*1000 = 333,333Hz)
Use Excel or another tool to calculate the values ​​for 100ms, 1000ms, ....

From my point of view, it makes no sense to carry out the acquirement directly after the start. And a freq. cant be calculated with 1x and duration of 0ms. ((1/0)*1000 = ?)
So maybe try to Change in Setup:
Code: [Select]

AcquireSensorDataNextMicros = init_time + acquireSensorDataDelayMicros;


It is difficult to reach an exact frequency of 333.333hz when the time is variable. The duration can be 999ms, 1000ms or 1001ms without changing the number of measurements. Or you make sure that the measurement is only finished in 3ms intervall.

Go Up