I am not ignoring post 2. Please refer to the last paragraph of post#3.
I need the RTC because I want to synchronize the data from multiple data collection stations. micros() and millis() return the number of microseconds and millisecond respectively from the time the Arduino board began running the current program. I cannot synchronize the data from different stations using micros() or millis() alone because different Arduino boards have different starting time to run their respective program.
Thanks for the correction. Just to double check. By "an event occurred", in this case we are talking about the fall of a square pulse from the SQE pin?
For the timestamp variable, what type is appropriate? Currently I have:
volatile unsigned long
for all time related variables.
Something to be aware of, the millis() count will occasionally increment by 2mS instead of 1mS because the clock it is based on is not exactly 1000Hz ( dividing down the 16MHz oscillator by powers of 2 results in 976.5625Hz), requiring an occasional correction to maintain better long term accuracy.
You also will need a way to maintain your data sampling while data is being written to the SD card.
When 1Hz square clock is used, the difference/elapse between micros() and timestamp is usually between 1155 and 1170 msec. So when divided by 1000, it gives 1. I wonder if it is better to divide by 1000 or 1000.00
When a 1KHz square clock is used, the difference/elapse is usually between 200 to 800 msec.
Do these numbers make sense?
So 1Hz clock generally gives more elapse time (above 1000 msec) than a 1KHz clock (less than 800 msec). because lower frequency means the timestamp is updated less often. On the other hand, the variation of 1Hz clock is less (15 vs 600).
I am not sure if I am getting the correct time down to msec.
If I lower my requirement to say, sampling at 50Hz (20msec), can this Adalogger (with RTC PCF8523 chip) get the job done? I think even if I buy a DS3231, its resolution is also 1 second.
Could you please clarify what you mean by "You also will need a way to maintain your data sampling while data is being written to the SD card."?
Right now, I am trying to get the time down to msec working. Then, I will work on getting the data at a user specified sampling rate.
Writing data to an SD card can take much longer than 1mS, although I'm not familiar with the Adalogger board so not sure how fast the SPI bus runs on that. There are some examples in the SdFat library for low latency logging that shows techniques for reducing the delays associated with the SD card such as wear leveling and having to erase before writing.
If timing down to the millisecond between stations is needed, you may want to use a GPS based clock.
May have missed it, did you mention what type sensor you are reading the data from?
Thanks. The SPI in Feather ESP32-S3 can handle only hundreds of kB/s. The suggestion on using a GPS based clock could be good. Is it easy to get the time in msec resolution once I have it? I plan to use ananlog sensor.
The unsigned long is appropriate.
The volatile qualifier is only needed in the case where the variable is used in something like an interrupt service routine (ISR). So whether or not the use of the qualifier is appropriate depends on the specifics of your code.
Good observation. I was simply adapting the code you had provided. However, if you are using this to measure durations of only a few milliseconds, then it seems best to simply use the more appropriate microseconds unit and remove the millisecond conversion code altogether.
Do you mean this?
When using the Adalogger, what is the highest sampling rate I can get?
Certainly taking a sample after each second is not high enough.
Yes.
I don't know the answer to that. My recommendation is to just give it a try. Remove any interval control code from your sketch, so that the program will just sample as fast as possible, then check to see how many samples it gathered over the session.
You might find that the rate is sufficient for your needs. If not, you can share the full sketch program here. The forum helpers may be able to suggest adjustments you can make to increase the rate.
Thank you. If I recall correctly, some of the numbers posted in Post#14 came from this program. I know that turning off the Serial.print statements will make things run faster.
Ultimately, I want something like:
0 msec corresponding output from sensor
1 msec corresponding output from sensor
2 msec corresponding output from sensor
:
1000 msec corresponding output from sensor
:
10000 msec corresponding output from sensor
I read that the Adalogger with RTC cannot produce such high time resolution. It missed some of the changes in sensory inputs.
// Add sd card
#include <Wire.h>
#include "RTClib.h"
#include <SPI.h>
#include "FS.h" // File system library
#include <SD.h> // SD card library
#define SD_CS 10 // For SD card
#define RECORD_DURATION 10000 // Save data for 10 seconds.
#define SD_MOSI 35
#define SD_MISO 37
#define SD_SCLK 36
#define SD_CS 10
File myFile;
// Keep track of number of seconds to run before stopping the program
int count_seconds = 0;
RTC_PCF8523 rtc;
const int SQW_PIN = 9; // Interrupt pin
volatile unsigned long timestamp;
volatile unsigned long current_time_in_micros;
volatile unsigned long elapsed;
volatile bool new_second = false;
// Interrupt Service Routine
void IRAM_ATTR onSecond() {
timestamp = micros();
new_second = true;
}
void setup() {
Serial.begin(115200);
while (!Serial)
{
// Wait for serial port to connect. Needed for native USB port only
;
}
Serial.println("Setup start");
// ------------------ SD CARD ----------------------
SPI.begin(SD_SCLK, SD_MISO, SD_MOSI, SD_CS);
if (!SD.begin(SD_CS))
{
Serial.println("SD Card MOUNT FAIL");
}
else
{
Serial.println("SD Card MOUNT SUCCESS");
myFile = SD.open("/data.txt", FILE_WRITE);
}
if (!rtc.begin()) {
Serial.println("Couldn't find RTC");
while (1);
}
// Configure SQW pin for 1KHz
//https://adafruit.github.io/RTClib/html/class_r_t_c___p_c_f8523.html#afac1866e69541cfb93251dee683f0e63
rtc.writeSqwPinMode(PCF8523_SquareWave1kHz);
//rtc.writeSqwPinMode(PCF8523_SquareWave4kHz);
//rtc.writeSqwPinMode(PCF8523_SquareWave1HZ);
//rtc.writeSqwPinMode(PCF8523_SquareWave32kHz);
//rtc.writeSqwPinMode(PCF8523_SquareWave8kHz);
pinMode(SQW_PIN, INPUT_PULLUP);
attachInterrupt(digitalPinToInterrupt(SQW_PIN), onSecond, FALLING);
if (! rtc.initialized() || rtc.lostPower()) {
Serial.println("RTC is NOT initialized, let's set the time!");
// When time needs to be set on a new device, or after a power loss, the
// following line sets the RTC to the date & time this sketch was compiled
rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
// This line sets the RTC with an explicit date & time, for example to set
// January 21, 2014 at 3am you would call:
// rtc.adjust(DateTime(2014, 1, 21, 3, 0, 0));
//
// Note: allow 2 seconds after inserting battery or applying external power
// without battery before calling adjust(). This gives the PCF8523's
// crystal oscillator time to stabilize. If you call adjust() very quickly
// after the RTC is powered, lostPower() may still return true.
}
// When time needs to be re-set on a previously configured device, the
// following line sets the RTC to the date & time this sketch was compiled
// rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
// This line sets the RTC with an explicit date & time, for example to set
// January 21, 2014 at 3am you would call:
// rtc.adjust(DateTime(2014, 1, 21, 3, 0, 0));
// When the RTC was stopped and stays connected to the battery, it has
// to be restarted by clearing the STOP bit. Let's do this to ensure
// the RTC is running.
rtc.start();
// The PCF8523 can be calibrated for:
// - Aging adjustment
// - Temperature compensation
// - Accuracy tuning
// The offset mode to use, once every two hours or once every minute.
// The offset Offset value from -64 to +63. See the Application Note for calculation of offset values.
// https://www.nxp.com/docs/en/application-note/AN11247.pdf
// The deviation in parts per million can be calculated over a period of observation. Both the drift (which can be negative)
// and the observation period must be in seconds. For accuracy the variation should be observed over about 1 week.
// Note: any previous calibration should cancelled prior to any new observation period.
// Example - RTC gaining 43 seconds in 1 week
float drift = 43; // seconds plus or minus over oservation period - set to 0 to cancel previous calibration.
float period_sec = (7 * 86400); // total obsevation period in seconds (86400 = seconds in 1 day: 7 days = (7 * 86400) seconds )
float deviation_ppm = (drift / period_sec * 1000000); // deviation in parts per million (μs)
float drift_unit = 4.34; // use with offset mode PCF8523_TwoHours
// float drift_unit = 4.069; //For corrections every min the drift_unit is 4.069 ppm (use with offset mode PCF8523_OneMinute)
int offset = round(deviation_ppm / drift_unit);
// rtc.calibrate(PCF8523_TwoHours, offset); // Un-comment to perform calibration once drift (seconds) and observation period (seconds) are correct
// rtc.calibrate(PCF8523_TwoHours, 0); // Un-comment to cancel previous calibration
Serial.print("Offset is "); Serial.println(offset); // Print to control offset
}
void loop() {
if (new_second) {
DateTime now = rtc.now();
new_second = false; // Reset flag
current_time_in_micros = micros();
elapsed = (current_time_in_micros-timestamp)/1000;
//elapsed = (current_time_in_micros-timestamp);
// Print timestamp with approximate milliseconds
Serial.print(now.year(), DEC);
Serial.print('/');
Serial.print(now.month(), DEC);
Serial.print('/');
Serial.print(now.day(), DEC);
Serial.print(" ");
Serial.print(now.hour(), DEC);
Serial.print(':');
Serial.print(now.minute(), DEC);
Serial.print(':');
Serial.print(now.second(), DEC);
Serial.print("micros(): ");
Serial.print(current_time_in_micros);
Serial.print(" - ");
Serial.print("timestamp: ");
Serial.println(timestamp);
Serial.print(" = elapsed: ");
// Calculate milliseconds elapsed since last 1KHz edge
Serial.print(elapsed);
Serial.println(" ms");
// Write data to file
myFile.print(now.hour(), DEC);
myFile.print(':');
myFile.print(now.minute(), DEC);
myFile.print(':');
myFile.print(now.second(), DEC);
myFile.print(':');
myFile.print(current_time_in_micros);
myFile.print(':');
myFile.println(timestamp);
myFile.print(':');
myFile.println(elapsed);
myFile.flush();
if (count_seconds > RECORD_DURATION)
{
myFile.close();
}
else
{
;
//count_seconds++;
}
count_seconds++;
}
//delay(10);
}
Please always do an auto format on your code before posting it to Arduino Forum. You can do that by selecting Tools > Auto Format from the Arduino IDE menus. The code you posted is very messy. Posting unformatted code is disrespectful to the forum helpers. Unformatted code is difficult to read, and so may result in the helpers being less effective in providing the assistance you are requesting.
You should also be formatting your code even when you aren't sharing it with others as this will allow you to more efficiently read and understand the structure.
You did not do what I suggested:
You have your sketch configured so that it only writes data on an interval. If your question is "what is the highest sampling rate?", then why are you imposing a limit on the rate?
The other problem is that your code doesn't sample anything. It could be useful to establish a baseline theoretical maximum rate without any overhead for reading the sensor, so it is reasonable to check the results from the code as it is. However, if you want to determine the true maximum sampling rate then you must include the sensor reading code in there as well.
Your program structure is incorrect here. Your program will continue to attempt logging even after RECORD_DURATION has been reached, and thus to write to myFile even after it has been closed.
The OP appears to have two threads on the Adafruit forum, same topic, also seeming to go nowhere.
Do you think they recommended the wrong products? The first guy said it could do 1msec but the second guy said no and then backtrack a bit. Now we know GPS does not work indoors so that is out of question.
I did not use that program to answer this question. I just asked if any forum user knows about that. That is why at the beginning of the code, I mentioned that some of the numbers posted in Post#14 came from it. It was just an attempt to sample the data every 1msec which I failed. When I was looking for sample codes to do that, google gave me one. The posted code is the one I modified from it.
No, you have not been misinformed. You are either misinterpreting the comments, or are not paying attention, or both.
Go back and read those threads carefully. It would also help to provide some actually useful information about your project.
If GPS requires an open space above it to work well, it is a problem.
I mentioned GPS in this discussion because there appeared to be a need to synchronize the data sampling between multiple stations. There is no way to easily synchronize multiple RTCs to within a millisecond of each other, and if you did accomplish that they would still drift out of that tolerance within a short amount of time. The GPS PPS signal can maintain a synchronization within 10's of nanoseconds. It was not know at the time that this project would be unable to use GPS because of the location.
Arduino is about DIY. I was trying to help you to obtain the information yourself. You could have spent a few minutes performing a simple experiment to get a verified answer to the question.
Instead of doing that, you are still waiting for someone to spoon feed you an answer to your ambiguous question a day later.