Updating time with NTPclient on startup

I have a project that is essentially a clock with some additional features. I am using an ESP8266 and and RTC to store the current time when the project is powered off. The ESP8266 connects to the Internet at startup using with WiFi. This all works fine, but I wanted to get the current time using NTP and synchronise the RTC to it. Eventually my plan is to add a radio clock to it to pick up the time signal from Anthorn. For now, I would like to get the time from NTP at startup. I found that using just one call to .update() in void startup() does not work. I experimented with various delays but did not find a reliable solution. I eventually found the following snippet online:

while(!timeClient.update()){
    timeClient.forceUpdate();
 }

So this is essentially if update() returns false, it will force an update until .update() succeeds. I have this within a statement that first sets up the WiFi:

  // WiFi setup
  Serial.print("MAC address: ");
  Serial.println(WiFi.macAddress());
  if (startWiFi(essid, psks)) {
    ipaddr = WiFi.localIP();
    Serial.print(F("IP address: "));
    sprintf(ipaddrstr, "%d.%d.%d.%d", ipaddr[0], ipaddr[1], ipaddr[2], ipaddr[3]);
    Serial.println(ipaddrstr);

    // Get time from NTP server
    timeClient.begin();
    Serial.println(F("Getting time from NTP server..."));
    while(!timeClient.update()){
      timeClient.forceUpdate();
    }
    Serial.println(F("Done."));
    NTPenabled = true;
    getCurrentTimeStr(timeStr);
    updateDisplayedTime(timeStr);
    getCurrentDateStr(dateStr);
    updateDisplayedDate(dateStr);
    if (RTCenabled) updateRTCTime();

  }else{
    Serial.println(F("WiFi not connected!"));
  }

At present, the code sends the status messages to serial to indicate whether it has succeeded or failed. These will eventually be removed or turned into debug messages which can be turned on or off. After the time has been updated via NTPclient, the program then proceeds to update the clock and display the current time.

This initially worked fine, but lately, it hangs quite frequently preventing the rest of the code from running. It is easy to see why because if timeClient.update() never returns true, the loop never ends and the messages show as follows:

20x4 display
MAC address: 44:17:93:10:F8:91
Starting WiFi client .......
IP address: 192.168.2.177
Getting time from NTP server...

The questions I have are why has it suddenly started failing? Was it a fortunate happenstance that I never saw it fail previously? Perhaps more importantly, is there a better to reliably check NTP time on startup before going into the main loop?

Which NTP server is the sketch getting the time from ?

Sorry, yes, probably should have included the following:

// NTP service
WiFiUDP ntpService;
NTPClient timeClient(ntpService, "europe.pool.ntp.org", 3600, 60000);
bool NTPenabled = false;
// United Kingdom (London, Belfast)
TimeChangeRule BST = {"BST", Last, Sun, Mar, 1, 60};        // British Summer Time
TimeChangeRule GMT = {"GMT", Last, Sun, Oct, 2, 0};         // Standard Time
Timezone UK(BST, GMT);
time_t timenow;

In answer to your question then, its europe.pool.ntp.org. Due to other commitments, the project hasn't been looked at for at least a couple of months but I started working on it again just this week.

the esp8266 has NTP in SDK

  configTime(TIME_ZONE, "pool.ntp.org");
  time_t now = time(nullptr);
  while (now < SECS_YR_2000) {
    delay(100);
    now = time(nullptr);
  }
  setTime(now);
1 Like

Thank you. Does this return the time in 'now' in Epoch time?

I also ran into this error:

Arduino: 1.8.19 (Linux), Board: "Generic ESP8266 Module, 80 MHz, Flash, Disabled (new aborts on oom), Disabled, All SSL ciphers (most compatible), 32KB cache + 32KB IRAM (balanced), Use pgm_read macros for IRAM/PROGMEM, dtr (aka nodemcu), 26 MHz, 40MHz, DOUT (compatible), 1MB (FS:64KB OTA:~470KB), 2, nonos-sdk 2.2.1+100 (190703), v2 Lower Memory, Disabled, None, Only Sketch, 115200"

/home/johnc/Data/Data/Arduino/Calendar-Clock/Calendar-Clock.ino: In function 'void setup()':
Calendar-Clock:214:16: error: 'TIME_ZONE' was not declared in this scope
  214 |     configTime(TIME_ZONE, "europe.pool.ntp.org");
      |                ^~~~~~~~~
Calendar-Clock:218:11: error: assignment of function 'time_t now()'
  218 |       now = time(nullptr);
      |       ~~~~^~~~~~~~~~~~~~~
Multiple libraries were found for "WiFiUdp.h"
 Used: /home/johnc/.arduino15/packages/esp8266/hardware/esp8266/3.0.2/libraries/ESP8266WiFi
 Not used: /opt/arduino/libraries/WiFi
exit status 1
'TIME_ZONE' was not declared in this scope

I don't know why I am getting multiple libraries for WiFiUdp.h, but its the TIME_ZONE error that I am interested in. Do I need to define TIME_ZONE somewhere? I notice in another example that configTime looks like this:

configTime(5*3600, 0, "pool.ntp.org");

It would be useful to know what the numbers mean. I am still researching that but getting rather confused. For example, in an interesting thread here it is suggested that one should never set the system time with a timezone:

In the issue he logged he suggests it should be:

configTime(0, 0, "pool.ntp.org");

so as to set the time to UTC, the time zone offset then being calculated from this to perform local timed actions. Yet there are plenty of examples such as the one above with various parameters being used. I still haven't found where those parameters are documented.

5*3600 = GMT +5 in seconds

includes and timezone

#include <sntp.h>
#include <TZ.h>

#define TIME_ZONE TZ_Europe_London

my clock project

1 Like

b707, thank you for that. I am still trying to get my head around what this guy is saying in those threads. For now though, that will suffice. I am in the UK and we are presently in BST, so GMT+1.

The lines below in my code which were based on the documentation of the Timezone library do work and affect the timezone when using NTPclient, even though they are used BEFORE the NTPclient.update() call is made. Quite how do they interact with NTPclient? Do they work with configTime()?

TimeChangeRule BST = {"BST", Last, Sun, Mar, 1, 60};        // British Summer Time
TimeChangeRule GMT = {"GMT", Last, Sun, Oct, 2, 0};         // Standard Time
Timezone UK(BST, GMT);

For now I am just trying to get to a point where I can test the code suggested by juraj so I just changed the example he kindly posted to:

    configTime(0,0, "europe.pool.ntp.org");
    time_t timenow = time(nullptr);
    while (timenow < SECS_YR_2000) {
      delay(100);
      timenow = time(nullptr);
    }
    setTime(timenow);

I changed the variable name 'now' to 'timenow' because apparently 'now' is a keyword as it comes up in red.

There can be quite a bit of delay sometimes, but it does complete and program execution moves on, however the time is completely wrong at the moment. Currently showing:

01:02:xx Thur 01/01/2066 !

where xx is counting seconds. I am trying to figure out what is going on....

UPDATE: Juraj, thank you for your further code example. Sorry I didn't spot it before posting this.

Why do some examples do this:

time_t now = time(nullptr);

whereas others do this:

time(&now);

?

Is it the configTime() or the time() command that calls the NTP process?

why not take a look to sourcecode of time() function to figure out what the parameter does?

1 Like

Been trying to find it for the last hour.... Just found this, not sure if its the right one:

I can see there is a related time and date structure and it interacts with POSIX, but I am struggling to understand much else unfortunately.

it's not a sourcecode in your link, it just a header

time_t A = time(&var) - returns the current system time as time_t value and store it in variable var, if the &var is not NULL
(System time - Wikipedia)

1 Like

b707, thank you for that explanation. That one makes sense. Not sure what the one with the NULL pointer is supposed to do. Where is it going to place the time? Given:

time_t A = time(nullptr);

I would expect that to return nothing in the variable A.

Having researched for the last couple of hours, here is what I think I understand so far:

Set the timezone, NTP server and contact get time:
configTime(gmtOffset_sec, daylightOffset_sec, ntpServer);

Set the timezone to GMT0 (i.e. UTC time), NTP server and contact get UTC time:
configTime(0, 0, ntpServer);

Sets the system? clock to the timezone in "TZ"?
setenv("TZ",timezone.c_str(),1);
tzset();

time_t type is a time storing type, where 0 = 01/01/1970 at 00:00:00 and it is a 4 byte value

Create a time_t variable:
time_t timenow;

Stores the current system time in timenow
time_t timenow = time(&timenow);

Same as above?
time_t timenow = time(nullptr);
or;
time(&now);

???:
settime(now);

Create structure to hold each time and date element with easy access for printing, displaying etc
struct tm timeinfo;

Set parameters in timeinfo to the local time:
timeinfo = localtime(&now)

Set parameters in timeinfo to the GMT time or system time?
timeinfo = gmtime(&now)

Put local time into timeinfo?
getLocalTime(&timeinfo)
What is the sense of localtime here? Presumably local system time? (as opposed to time mofified for timezone?)

Set timezone from where? Same as setenv()/tzset()?
setTimezone(timezone);

I don't just want to take an example and plug it into my code. I want to understand what each command is doing, but there are so many sources and variations that it is difficult to know what has been applied correctly and what hasn't. I still can't find the corresponding source code to time.h.

what problem do you try to solve?

Originally, why NTPclient was getting stuck. However, since you posted your alternative code snippet, which I very much appreciate, I have been trying to understand the ESP8266/32 built-in calls to time and time zone functions so as to implement proper time zone handling and conversion from epoch time to day/month/year and hrs/min/sec. The above was my attempt to catalog what I have learned while doing my research earlier today. As it is evident from my list, I still have gaps in my understanding of some of the functions so if anyone can help me fill those in, that would help me a great deal.

My tests show that your method of getting NTP time works and does not hang. I now need to replace other calls to NTPclient.h and Timezone.h related functions with built-in ESP8266 functions. Before I can do that however, I need to understand what they do, exactly what information is being stored, in what format and where, and how these time and timezone functions interact with each other. It would also help to understand why some examples use one form of syntax while some use another. With that understanding I then be in a position to convert, write and enhance my own code rather than blindly follow examples. I had hoped to find some more detailed documentation, but the best source I have so far is that time.h file.

the SDK handles the timezone correctly based on the TC constant provided in configTime. it handles the daylight saving switch too.

I linked may clock project so you can see there how to get local time values.

    time_t t = time(nullptr);
    struct tm *tm = localtime(&t);
    if (tm->tm_year >= 120) { // check if we have a valid time
      byte minute = tm->tm_min;
      byte hour = tm->tm_hour;
1 Like

Thank you for confirming that about the SDK. I grasped a good deal about the methods and functions from your project so am really grateful to you for posting that.

Many online examples seem to use NTPclient which is why I used it in my project, although from what I have learned here, this library is not really needed. Having previously used NTPclient and Timezone libraries in my project, I am now having to differentiate between the methods provided by these libraries and the available built-in methods. Functions where I have used the library methods will need to be rewritten to use the built-in methods. I am working my way though that and it should simplify things once its done.

https://github.com/esp8266/Arduino/blob/master/libraries/esp8266/examples/NTP-TZ-DST/NTP-TZ-DST.ino

the SDK updates the time retrieved from NTP into the internal RTC. SDK updates the time from the NTP server periodically. the update interval is set by redefining the function sntp_update_delay_MS_rfc_not_less_than_15000

 uint32_t sntp_update_delay_MS_rfc_not_less_than_15000 () {
    return 3600000; // 1 hour 
}

every time the time is successfully updated from NTP, the function set with settimeofday_cb is called.

what else?

Yes, that makes sense although that is possibly the longest function name I have ever seen!. The ESP8266 and ESP32 have their own built in RTC so it would make sense for the time fetched from NTP to update the RTC.

I have made some progress with my project and replaced all of the NTPclient code with equivalent ESP built-in methods. It works except for the clock display which is one hour out. NTP gets the correct time and it is being modified for the timezone which I can print it to the Serial Monitor by deriving its elements from localtime():

void printTimeStamp(time_t rawtime){
  struct tm * timeinfo;
  char timeanddate[72];
  timeinfo = localtime(&rawtime);
  uint8_t wd = ((rawtime / 86400) + 4) % 7;
  sprintf(timeanddate, "%02d/%02d/%04d %02d:%02d:%02d",
      timeinfo->tm_mday,
      timeinfo->tm_mon,
      timeinfo->tm_year+1900,
      timeinfo->tm_hour,
      timeinfo->tm_min,
      timeinfo->tm_sec);
  Serial.println(timeanddate);
}

I do that in void startup() after getting the NTP time and setting the timezone and I get the correct time. However, the display always shows the UTC rather than GMT0BST, i.e. GMT + 1. I am then reading the RTC to update the clock. The function is structured such that if for some reason the RTC was not available it would use NTP time. Still trying to figure that part out.

This is how I update the RTC from NTP:

void loop() {

  bool chkdate = false;
  static int adcval = 0;

  // Turn on display backlight if too dark
  adcval = analogRead(LDR);
  if ( adcval<(ADCMAX/1.5) ) {
    lcd.setBacklight(255);
  }else{
    lcd.setBacklight(0);
  }

//  Serial.print(F("ADC value: "));
//  Serial.println(adcval);

  chkdate = getCurrentTimeStr(timeStr);
//  Serial.println(timeStr);
  updateDisplayedTime(timeStr);
/*
  if(chkdate) {
    if (NTPenabled && RTCenabled) {
      updateRTCTime();
    }
    getCurrentDateStr(dateStr);
    updateDisplayedDate(dateStr);
  }
 */

  // Check for events
/*
//  checkEvents();
  if (eventCnt>0) {
    updateDisplayedEvent();
  }else{
    displayNoEventMsg();
  }
*/
//  delay(200);

  delay(1000);
  
}

void getNTPtime(){
  // Get UTC time from NTP server
  configTime(0,0, "europe.pool.ntp.org");
  time_t timenow = time(nullptr);
  // Wait until we have a sensible time
  while (timenow < 946684800UL) { // the time at the start of y2k)
    delay(100);
    timenow = time(nullptr);
  }
  // Set the system time
  setTime(timenow);
}

void setTimezone(String timezone){
  Serial.printf("  Setting Timezone to %s\n",timezone.c_str());
  setenv("TZ",timezone.c_str(),1);  //  Now adjust the TZ.  Clock settings are adjusted to show the new local time
  tzset();
}

void updateRTCTime() {
  struct tm * timeinfo;
  getNTPtime();
  time_t rawtime = time(&rawtime);

  // void setEpoch(time_t epoch = 0, bool flag_localtime = false);
  // epoch = UnixTime and starts at 01.01.1970 00:00:00
  ds3231.setEpoch(rawtime, false);
}

This is how I obtain the time string from RTC or NTP for the clock:

bool getCurrentTimeStr(char * timestr){
  uint8_t curhrs = 0;
  uint8_t curmin = 0;
  uint8_t cursec = 0;
  time_t rawtime;
  struct tm * timeinfo;

  if (RTCenabled) { // Get time from RTC
    /*
    curhrs = ds3231.getHour(h12Flag, pmFlag);
    curmin = ds3231.getMinute();
    cursec = ds3231.getSecond();
    */
    // Get Unix time from RTC
    DateTime rtcnow = ds3231now.now();
    rawtime = rtcnow.unixtime();
    // Set time to UTC
    setTime(rawtime);
    // set the system time to UTC
    timeval tv = {rawtime, 0};
    settimeofday(&tv, nullptr);
  }else if (NTPenabled) { // Get time from local epoch updated by NTP server
    getNTPtime();
  }

//  setTimezone(TIME_ZONE);

  // Get system time
  rawtime = time(nullptr);
  // Get time elements
  timeinfo = localtime(&rawtime);
  curhrs = timeinfo->tm_hour;
  curmin = timeinfo->tm_min;
  cursec = timeinfo->tm_sec;

  sprintf(timestr, "%02d:%02d:%02d", \
    curhrs, \
    curmin, \
    cursec );

  // Flag a check of the date ?
  if ( (curhrs==12) || (curhrs==24) ) {
    if ( (curmin==0) && (cursec==0) ) {
      return true;
    }
  }
  return false;
}

The date string is obtained in a similar manner.

btw: the timezones in TZ.h are not just a number. it is a string with zone definition.
#define TZ_Europe_London PSTR("GMT0BST,M3.5.0/1,M10.5.0")
so it contains info about summer time