DS3231 vs. System Clock and auto-time sync

This issue has bee perplexing me for several years, today I decided to explore it further. You see, I need a reasonable accuracy on a long term clock for my chicken coop automation project. A minute or so a year is not an issue, but I often see 5 or more minutes a month in time errors. If I reset the RTC, all is good, but I can see the drift. So yesterday and today I spent large parts of the day researching to figure out the correct way to sync the RTC with the system clock. I learned this is supposed to happen automatically every 5 minutes with the library(ies) I am using. DS1307RTC - PJRC, and TimeLib. I wanted to see the sync actually happen, so I stuck a Serial.println into the timelib .cpp file. It simply prints "sync" when the setSyncProvider is called. (There is also a function to allow customization of the sync, but I am running the default 300 Sec) The serial monitor confirms the print during initialization, and then that is that...never see it again!
So, I did more searching, and found Jack Christensen's DS3232RTC lib. One of the examples compares the drift of the millis timer to the RTC. I had to move my exploration to a Duelmilanove I used for older libs, and found his code quite useful as I was able to set my DS3231 accurately with the serial example. Then I noticed the mcu-clock-drift example, and loaded it up. It reports that in 15 minutes I lost 2 seconds. The same sync annotated library is used, and no sync other than the initial has shown in hour or so this has been running...I also have not had another lost time report. No other hardware but the RTC module is running on this board. My long term board out in the coop is an Uno. And I am developing on a Nano_Every, (that is why the DS3232 doesn't work...it falls into INCOMPATIBLE libs)
In keeping with the forum rules, a copy of the running code is posted below:

// Arduino DS3232RTC Library
// https://github.com/JChristensen/DS3232RTC
// Copyright (C) 2018 by Jack Christensen and licensed under
// GNU GPL v3.0, https://www.gnu.org/licenses/gpl.html
//
// Example sketch to demonstrate microcontroller clock drift vs. RTC.
// The MCU's time of day is synchronized with the RTC every five minutes (by default).
// Between syncs, the MCU's time of day is goverened by its system clock (which
// also is used for millis). Because the MCU system clock and the RTC cannot be
// expected to run at identical rates, discontinuities in the MCU's time of day
// will occur when its time is synchronized with the RTC.
//
// We can see the drift and discontinuities by printing both millis and
// the time of day together.
//
// Jack Christensen 27Dec2018

#include <DS3232RTC.h>      // https://github.com/JChristensen/DS3232RTC

void setup()
{
    Serial.begin(115200);
    Serial.println(F("\n" __FILE__ " " __DATE__ " " __TIME__));
    setSyncProvider(RTC.get);   // the function to get the time from the RTC
    Serial.print(F("RTC sync "));
    if (timeStatus() == timeSet)
        Serial.println(F("OK"));
    else
        Serial.println(F("FAIL!"));
}

// when the MCU's time of day changes, if it did not move forward
// by one second, or if millis changed by something other than about
// one second, then print the millis and time values before and after.
void loop()
{
    static uint32_t msLast;
    static time_t tLast;
    uint32_t ms = millis();
    time_t t = now();
    if (t != tLast)
    {
        if (t - tLast != 1 || ms - msLast > 1010 || ms - msLast < 990)
        {
            Serial.print(msLast);
            Serial.print(' ');
            Serial.print(ms);
            Serial.print(' ');
            printTime(tLast);
            Serial.print(' ');
            printTime(t);
            Serial.println();
        }
        tLast = t;
        msLast = ms;
    }
}

// format and print a time_t value
void printTime(time_t t)
{
    if (t > 0)
    {
        char buf[25];
        char m[4];    // temporary storage for month string (DateStrings.cpp uses shared buffer)
        strcpy(m, monthShortStr(month(t)));
        sprintf(buf, "%.2d:%.2d:%.2d %.4d-%.2d-%.2d",
            hour(t), minute(t), second(t), year(t), month(t), day(t));
        Serial.print(buf);
    }
    else
    {
        Serial.print(F("Start"));
    }
}

Here is the serial monitor output generated thus far. under the path line, there is "sync", that is generated from my addition to Time.cpp. (The other "sync" is part of the program. Below is the last two functions in the library in which I added the Serial.print for debugging.

void setSyncProvider( getExternalTime getTimeFunction){
  getTimePtr = getTimeFunction;  
  nextSyncTime = sysTime;
  now(); // this will sync the clock
  Serial.println("sync"); //added by me
}

void setSyncInterval(time_t interval){ // set the number of seconds between re-sync
  syncInterval = (uint32_t)interval;
  nextSyncTime = sysTime + syncInterval;
}

Conveniently, these are the last two listed functions in the file, should anyone want to look that deep.

My question of course is what do I need to do to get the RTC to update the system clock? I thought I might set the "setSyncProvider" into a millis() loop next to see what that causes, as I know it runs the update.

Any suggestions?

BTW, I want to thank Paul and Jack for all their work, not only on RTCs, but their activity all around! Great work and help!! Also, thanks to all those active here on the Forum! Happy New Year!

-fab

There is no "of course" about it. If you have an RTC, why not just use that? And what is a "system clock" anyway? It's hard to see what you are doing, or what you want to do, but a DS1307 is probably a bad choice out in the coop as it has no temperature compensation, although I don't suppose the chooks will be too upset about 5 minutes/month.

The DS3231 you mention in the heading is accurate and reliable, and should be fine for your needs. If it is incompatible with the Nano Every, refraining from developing thereon is probably a legitimate option. The DS3231 is fine on Uno/Nano/Pro Mini, and is just about everybody's favourite RTC.

No point in banging on the I2C bus every time you want to know the time. That's why PJRC time library using the "system clock" (it's based on millis() on AVR boards) to report the time and periodically trues it up with the RTC via setSyncProvider().

@fabelizer, you only get one 'sync' message because setSyncProvider() is only called once. That function doesn't do the actual sync. As the name suggests, it sets the provider. The provider is what is called periodically to the the sync.

I can't tell you why that doesn't appear to be working. If it were me, I'd write my own sync provider function. In that I'd call RTC.get() and intercept the returned value. Print that to see if it makes sense then return it to the Time library that called your sync provider.

Thanks both.
@Nick_Pyner I am using the DS3231. The DS1307 lib work with IT too. (Just no alarms, temperature, etc. that the 1307 doesn't have). The chooks HATE a late exit! Also, the full project will be using I2C for several things, a display, a port expander, and output control. I don't want continuous RTC traffic on it too.

@gfvalvo Yes, that is what I thought until I found the information regarding the 300 second sync. Tomorrow I'll toss the setSyncProvider(RTC.get) into a BWOD millis timer and see what happens. None of the examples, or other code I've read shows the need of that method, so it seems plausible that it should automatically sync. I am not yet skilled enough to understand the entire library, but I'm working on it!

Thanks Again!
-fab

That makes no sense and has nothing to do with my suggestion. If you do what I recommend, you'll see the snyc provider being called.

Also, shorten the 300 second so you don't have to wait so long in between calls.

I do understand the entire library.

I understand what you are saying, but I have no idea how to implement that in code. I would think using the function provided and calling it periodically would work OK. It is well beyond me to "Print that to see if it makes sense then return it to the Time library that called your sync provider." I realize the Time.cpp called the sync provider, but returning the value to it, without calling that function is what has me reeling! Printing it is not a problem. And I can use the existing function to help me understand, but why not just call the provided function on schedule?

Not arguing here...trying to learn.
-fab

I am not yet skilled enough to understand the entire library, but I'm working on it!

Study the time library function now(). That is where the synchronization take place at the specified interval. getTimePtr() is calling the function parameter given to setSyncProvider().


time_t now() {
	// calculate number of seconds passed since last call to now()
  while (millis() - prevMillis >= 1000) {
		// millis() and prevMillis are both unsigned ints thus the subtraction will always be the absolute value of the difference
    sysTime++;
    prevMillis += 1000;	
#ifdef TIME_DRIFT_INFO
    sysUnsyncedTime++; // this can be compared to the synced time to measure long term drift     
#endif
  }
  if (nextSyncTime <= sysTime) {
    if (getTimePtr != 0) {
      time_t t = getTimePtr();
      if (t != 0) {
        setTime(t);
      } else {
        nextSyncTime = sysTime + syncInterval;
        Status = (Status == timeNotSet) ?  timeNotSet : timeNeedsSync;
      }
    }
  }  
  return (time_t)sysTime;
}
void setSyncProvider( getExternalTime getTimeFunction){
  getTimePtr = getTimeFunction;  
  nextSyncTime = sysTime;
  now(); // this will sync the clock
}

It doesn't sound like you really are understanding what the sync provider function is doing, how the sync works and what @gfvalvo is saying.

When you call set sync provider. i.e. :

setSyncProvider(RTC.get);   // the function to get the time from the RTC

It tells the TimeLib library to periodically call that function, RTC.get() in this case, to update the internal time stamp inside the TimeLib library.
You only call setSyncProvider() once. The TimeLib library periodically calls the specified sync function. Your print message was inside setSyncProvider() so it only prints once and does not print when the sync provider function is actually called. i.e. nothing will print when RTC.get() is called from TimeLib

@gfvalvo is saying to write a wrapper function for the RTC.get() function so you can do other things when synchronization happens like print something each time it is called to see if the returned timestamps make sense.

i.e. create a wrapper function in your sketch

time_t mysync(void)
{
time_t t;
    t = RTC.get();
    Serial.print("TimeLib sync: ");
    Serial.println(t);
    return(t);
}

Then change the call set sync provider to this instead:
setSyncProvider(mysync);

This will cause the TimeLib library toperiodically call the mysync() function to sync the timestamp. And in this case the code will print the message and the timestamp each time the sync provider is called.

Keep in mind that since the TimeLib does not use interrupts, now() must be periodically called in order for the TimeLib to notice time passing and call the sync provider function.

--- bill

@bperrybap Thanks for the bump! I'll check that a bit deeper tomorrow. For the moment, I've pasted the function into Jack's example code and commented out what I had running this morning.

// Arduino DS3232RTC Library
// https://github.com/JChristensen/DS3232RTC
// Copyright (C) 2018 by Jack Christensen and licensed under
// GNU GPL v3.0, https://www.gnu.org/licenses/gpl.html
//
// Example sketch to demonstrate microcontroller clock drift vs. RTC.
// The MCU's time of day is synchronized with the RTC every five minutes (by default).
// Between syncs, the MCU's time of day is goverened by its system clock (which
// also is used for millis). Because the MCU system clock and the RTC cannot be
// expected to run at identical rates, discontinuities in the MCU's time of day
// will occur when its time is synchronized with the RTC.
//
// We can see the drift and discontinuities by printing both millis and
// the time of day together.
//
// Jack Christensen 27Dec2018

#include <DS3232RTC.h>      // https://github.com/JChristensen/DS3232RTC
unsigned long prevSync = 0;
//const unsigned long syncInterval = 75000;

void setup()
{
    Serial.begin(115200);
    Serial.println(F("\n" __FILE__ " " __DATE__ " " __TIME__));
    setSyncProvider(mysync);   // the function to get the time from the RTC
    Serial.print(F("RTC sync "));
    if (timeStatus() == timeSet)
        Serial.println(F("OK"));
    else
        Serial.println(F("FAIL!"));
}

// when the MCU's time of day changes, if it did not move forward
// by one second, or if millis changed by something other than about
// one second, then print the millis and time values before and after.
void loop()
{
    //unsigned long currentMillis = millis();

    static uint32_t msLast;
    static time_t tLast;
    uint32_t ms = millis();
    time_t t = now();
    if (t != tLast)
    {
        if (t - tLast != 1 || ms - msLast > 1010 || ms - msLast < 990)
        {
            Serial.print(msLast);
            Serial.print(' ');
            Serial.print(ms);
            Serial.print(' ');
            printTime(tLast);
            Serial.print(' ');
            printTime(t);
            Serial.println();
        }
        tLast = t;
        msLast = ms;
    }
   // if(currentMillis - prevSync >= syncInterval){
   //   setSyncProvider(RTC.get);
   //   prevSync = currentMillis;
   // }
}

// format and print a time_t value
void printTime(time_t t)
{
    if (t > 0)
    {
        char buf[25];
        char m[4];    // temporary storage for month string (DateStrings.cpp uses shared buffer)
        strcpy(m, monthShortStr(month(t)));
        sprintf(buf, "%.2d:%.2d:%.2d %.4d-%.2d-%.2d",
            hour(t), minute(t), second(t), year(t), month(t), day(t));
        Serial.print(buf);
    }
    else
    {
        Serial.print(F("Start"));
    }
}

time_t mysync(void)
{
time_t t;
    t = RTC.get();
    Serial.print("TimeLib sync: ");
    Serial.println(t);
    return(t);
}

Output:


/home/dave/Arduino/mcu_clock_drift_DAK/mcu_clock_drift_DAK.ino Jan  6 2022 01:31:01
TimeLib sync: 1641432643
sync
RTC sync OK
0 5 Start 01:30:43 2022-01-06
TimeLib sync: 1641432743
TimeLib sync: 1641432843
TimeLib sync: 1641432943
TimeLib sync: 1641433043
TimeLib sync: 1641433143

In TimeLib, I shortened the sync time to 100 seconds. (I can see in your t print that it is indeed 100, so that is great! I have not removed my print statement yet, and I am pretty worn from work today, so I;m done for the night.

I really appreciate your clarification and code. Tomorrow I'll let this run awhile, and hopefully see the millis error appear. It hasn't failed to show up on previous runs.
Thanks,
-fab

Why did you modify TimeLib instead of just calling setSyncInterval()?

It was previously done while experimenting. At the time, I had no visibility to any sync happening at all. Now, this morning, I am going to change it back, verify it changed, then use setSyncInterval to modify it again. Thanks for your help, I am sorry I did not get what you were trying to tell me!

-fab

Interestingly, the day I started this topic, I had reset my DS3231 via the SetSerial example for the DS3232 lib. I set it to the clock on the computer, as that is NTP synced, and also verified to my mobile. Very soon after set, it went 30 Sec slow (a day or so) and stayed that way two days. The only program run on the UNO is various iterations of the 'mcu_clock_drift' example I have shown in this topic, which has no facility to set the time on the RTC. As late as 1:00 AM EST today, it was 30 Sec slow. Now 10:00 AM EST, today it is 45 Sec slow to the computer? Baffling!

Remember, I engaged in this time issue because of an inaccurate clock on a previous iteration that is still running out in the coop. This is making no sense to me at the moment! It is a Chinese module, so I suppose it could be a forged chip marked DS3231N. I think I'll order one from Digikey and see if I am still having issues.

I have no idea why this happens. The RTC remains powered, the battery is at 3V, and is a standard CR2032. The recharge circuit has been disabled by removing the charge resistor, and it does remain set with no power applied.

-fab

From your posts it still isn't clear to me if you are using a DS1307 module or a DS3231 module. You seem to have some contradictory posts.
While a library may support either/or/or both h/w modules, it is the hardware that matters.
You want to use a DS3231 h/w module as that one is much more accurate than the DS1307 module.

--- bill

@bperrybap I'm using a DS3231 ZS-042 module. The one I was using had a problem with loosing time, which drove me into the libraries. I have substituted it for another of the same. I did that this morning and so far all is well.

Since everything "seemed" to be working fine on the 1st module, (e.g. time came back in proper format, could read temperature) I believed the clock chip was fine. So I began exploring code issues. In fact, it was NOT fine, and now I'm using another module that seems to be keeping time as it should. I am only about 8 hours into the test, so after it runs a few days, I'll feel more confident.

Thanks for helping,
-fab

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.