NTP delivers wrong date / unstable behaviour

I assembled a program to synchronize my RTC DS1307.

  • I start with an automatic update of the RTC: RTC.adjust(DateTime(DATE, TIME). This works fine;
  • I adjust the RTC date and time to wrong values;
  • I call a function Synchronize_RTC

After synchronizing the month, day, hour, minute and second are correct, but the year is 2046 in stead of 2016.
I can't find what is wrong.

/*
 Udp NTP Client
 Source: https://github.com/codebendercc/arduino-library-files/blob/master/libraries/Ethernet/examples/UdpNtpClient/UdpNtpClient.ino
 Get the time from a Network Time Protocol (NTP) time server
 Demonstrates use of UDP sendPacket and ReceivePacket 
 For more on NTP time servers and the messages needed to communicate with them, 
 see http://en.wikipedia.org/wiki/Network_Time_Protocol
 
 Warning: NTP Servers are subject to temporary failure or IP address change.
 Plese check http://tf.nist.gov/tf-cgi/servers.cgi if the time server used in the example didn't work.
 
 Created 4 Sep 2010 by Michael Margolis
 Modified 9 Apr 2012 by Tom Igoe
 Modified 25 March 2016 C.W.A. Baltus
 
 Hardware:
- Arduino Mega 2560 Rev.3
- Ethernet Shield W5100
- RTC DS1307

Software:
- Version 1.6.1

 */

//================================================================================================
//Libraries
#include <Wire.h>      //Enables I2C communication
#include <RTClib.h>
#include <SPI.h>         
#include <Ethernet.h>
#include <EthernetUdp.h>

RTC_DS1307 RTC;

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

unsigned int localPort = 8888;      // local port to listen for UDP packets

IPAddress timeServer(129, 6, 15, 28); 
//IPAddress timeServer(132, 163, 4, 101); // time-a.timefreq.bldrdoc.gov NTP server  //OKE 20160324
//IPAddress timeServer(132, 163, 4, 102); // time-b.timefreq.bldrdoc.gov NTP server   //OKE 20160324
//IPAddress timeServer(132, 163, 4, 103); // time-c.timefreq.bldrdoc.gov NTP server

const int NTP_PACKET_SIZE= 48; // NTP time stamp is in the first 48 bytes of the message
byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing packets 


// A UDP instance to let us send and receive packets over UDP
EthernetUDP Udp;

//==========================================================================
void setup() { 
  Serial.begin(9600); // Open serial communications and wait for port to open:
  Wire.begin();
  RTC.begin();
  
  if (! RTC.isrunning()) {
       Serial.println("RTC is NOT running!");
       // Synchronizing the RTC to the (date & time) of the PC
       RTC.adjust(DateTime(__DATE__, __TIME__));
       } 
  
  RTC.adjust(DateTime(__DATE__, __TIME__));
  DateTime now = RTC.now();
  Serial.print("Time after automatic update: ");
  Serial.print(now.year(), DEC);     Serial.print("-"); Serial.print(now.month(),DEC); Serial.print("-");  Serial.print(now.day(), DEC);       Serial.print("   ");
  Serial.print (now.hour(), DEC); Serial.print(":"); Serial.print(now.minute(), DEC);   Serial.print(":");  Serial.println (now.second(), DEC);
  
   // start Ethernet and UDP
  if (Ethernet.begin(mac) == 0) 
      {Serial.println("Failed to configure Ethernet using DHCP");  } 
      else 
    {Serial.println("Ethernet connection succesfuill");}
  
  Udp.begin(localPort);
  
  //Set manually a wrong time
  RTC.adjust(DateTime(2015,6,21,9,51,05));
  now = RTC.now();
  Serial.print ("Wrong time:                 ");
  Serial.print(now.year());     Serial.print("-"); Serial.print(now.month(),DEC); Serial.print("-");  Serial.print(now.day(), DEC);       Serial.print("   ");
  Serial.print (now.hour(), DEC); Serial.print(":"); Serial.print(now.minute(), DEC);   Serial.print(":");  Serial.println (now.second(), DEC);
  
  //Synchronize the RTC with NTP
  Synchronize_RTC();
  now = RTC.now();
  Serial.print ("Corrected time with NTP:    ");
  Serial.print(now.year());     Serial.print("-"); Serial.print(now.month(),DEC); Serial.print("-");  Serial.print(now.day(), DEC);       Serial.print("   ");
  Serial.print (now.hour(), DEC); Serial.print(":"); Serial.print(now.minute(), DEC);   Serial.print(":");  Serial.println (now.second(), DEC);
}

//==========================================================================
void loop(){
  
}

//==========================================================================
unsigned long sendNTPpacket(IPAddress& address) 
// send an NTP request to the time server at the given address, set all bytes in the buffer to 0
  {
  memset(packetBuffer, 0, NTP_PACKET_SIZE); 
  // Initialize values needed to form NTP request (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;            // Stratum, or type of clock
  packetBuffer[2] = 6;            // Polling Interval
  packetBuffer[3] = 0xEC;         // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12]  = 49; 
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;

  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp: 		   
  Udp.beginPacket(address, 123); //NTP requests are to port 123
  Udp.write(packetBuffer ,NTP_PACKET_SIZE);
  Udp.endPacket(); 
  }  //End sendNTPpacket

//============================================================================
void Synchronize_RTC()
  {
  sendNTPpacket(timeServer); // send an NTP packet to a time server

  // wait to see if a reply is available
  delay(1000); 
  if ( Udp.parsePacket() ) {  
      // We've received a packet, read the data from it
      Udp.read(packetBuffer,NTP_PACKET_SIZE);  // read the packet into the buffer
      //the timestamp starts at byte 40 of the received packet and is four bytes, or two words, long. First, esxtract the two words:

     // NTP contains four timestamps with an integer part and a fraction part. We only use the integer part here
      unsigned long t1, t2, t3, t4;
      t1 = t2 = t3 = t4 = 0;
      for (int i=0; i< 4; i++)  {
          t1 = t1 << 8 | packetBuffer[16+i];      
          t2 = t2 << 8 | packetBuffer[24+i];      
          t3 = t3 << 8 | packetBuffer[32+i];      
          t4 = t4 << 8 | packetBuffer[40+i];}
      //Serial.print(t1); Serial.print("   "); Serial.print(t2); Serial.print("   ");Serial.print(t3); Serial.print("   ");Serial.println(t4); 
      
      // part of the fractional part, could be 4 bytes but this is more precise than the 1307 RTC which has a precision of ONE second
      // in fact one byte is sufficient for 1307
      float f1,f2,f3,f4;
      f1 = ((long)packetBuffer[20] * 256 + packetBuffer[21]) / 65536.0;      
      f2 = ((long)packetBuffer[28] * 256 + packetBuffer[29]) / 65536.0;      
      f3 = ((long)packetBuffer[36] * 256 + packetBuffer[37]) / 65536.0;      
      f4 = ((long)packetBuffer[44] * 256 + packetBuffer[45]) / 65536.0;

      const unsigned long seventyYears = 2208988800UL;
      //const unsigned long seventyYears =0UL;
      t1 -= seventyYears;
      t2 -= seventyYears;
      t3 -= seventyYears;
      t4 -= seventyYears;
      
      // Adjust timezone and DST... in my case substract 4 hours for Chile Time or work in UTC?
      t4 += (2 * 3600L);     // Notice the L for long calculations!!
      t4 += 1;               // adjust the delay(1000) at begin of loop!
      if (f4 > 0.4) t4++;    // adjust fractional part, see above
      RTC.adjust(DateTime(t4));
  }  //End ParsePcket

} //End function Synchronize_RTC

The RTCLib appears to use the year 2000 as its epoch. You will have to change your code so that you adjust the date by subtracting oneHundredYears instead of seventyYears.

Pete

Thanks Pete, I changed the code.

const unsigned long HundredYears = 3155751673UL;

I tried to find out the number of seconds for 100 years for adjusting my RTC with NTP.
In the serial monitor ou can see that the time difference between the RTC-time (set with RTC.adjust(DateTime(DATE, TIME)):wink: and the NTP is not stable.

Some instructions give the wrong results.
Why? I tried allready to build in some delays.

/*
 Udp NTP Client
 Source: https://github.com/codebendercc/arduino-library-files/blob/master/libraries/Ethernet/examples/UdpNtpClient/UdpNtpClient.ino
 Get the time from a Network Time Protocol (NTP) time server
 Demonstrates use of UDP sendPacket and ReceivePacket
 For more on NTP time servers and the messages needed to communicate with them,
 see http://en.wikipedia.org/wiki/Network_Time_Protocol
 
 Warning: NTP Servers are subject to temporary failure or IP address change.
 Plese check http://tf.nist.gov/tf-cgi/servers.cgi if the time server used in the example didn't work.
 
 Created 4 Sep 2010 by Michael Margolis
 Modified 9 Apr 2012 by Tom Igoe
 Modified 25 March 2016 C.W.A. Baltus
 
 Hardware:
- Arduino Mega 2560 Rev.3
- Ethernet Shield W5100
- RTC DS1307

Software:
- Version 1.6.1

 */

//================================================================================================
//Libraries
#include <Wire.h>      //Enables I2C communication
#include <RTClib.h>
#include <SPI.h>         
#include <Ethernet.h>
#include <EthernetUdp.h>

RTC_DS1307 RTC;

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

unsigned int localPort = 8888;      // local port to listen for UDP packets
unsigned long start;
DateTime now;

IPAddress timeServer(129, 6, 15, 28);
//IPAddress timeServer(132, 163, 4, 101); // time-a.timefreq.bldrdoc.gov NTP server  //OKE 20160324
//IPAddress timeServer(132, 163, 4, 102); // time-b.timefreq.bldrdoc.gov NTP server   //OKE 20160324
//IPAddress timeServer(132, 163, 4, 103); // time-c.timefreq.bldrdoc.gov NTP server

const int NTP_PACKET_SIZE= 48; // NTP time stamp is in the first 48 bytes of the message
byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing packets


// A UDP instance to let us send and receive packets over UDP
EthernetUDP Udp;

//==========================================================================
void setup() {
  Serial.begin(9600); // Open serial communications and wait for port to open:
  Wire.begin();
  RTC.begin();
 
  /*if (! RTC.isrunning()) {
       Serial.println("RTC is NOT running!");
       // Synchronizing the RTC to the (date & time) of the PC
       RTC.adjust(DateTime(__DATE__, __TIME__));
       }*/
 
     // start Ethernet and UDP
  if (Ethernet.begin(mac) == 0)
      {Serial.println("Failed to configure Ethernet using DHCP");  }
      else
    {Serial.println("Ethernet connection succesfull");}
 
  Udp.begin(localPort);
  
  }  //End setup

//==========================================================================
void loop(){
  RTC.adjust(DateTime(__DATE__, __TIME__));
  delay(100);
  start = millis(); 
  now = RTC.now();
  Serial.print("RTC-time after synchr PC  ");
  Serial.print(now.year(), DEC);     Serial.print("-"); Serial.print(now.month(),DEC); Serial.print("-");  Serial.print(now.day(), DEC);       Serial.print("   ");
  Serial.print (now.hour(), DEC); Serial.print(":"); Serial.print(now.minute(), DEC);   Serial.print(":");  Serial.println (now.second(), DEC);
  Serial.print ("Tijd: "); Serial.print(millis()-start); Serial.print("\n");
  
  //Set the RTC wrong 
  RTC.adjust(DateTime(2015,6,21,9,51,05));
  delay(50);
  now = RTC.now();
  Serial.print ("Set RTC wrong:           ");
  Serial.print(now.year());     Serial.print("-"); Serial.print(now.month(),DEC); Serial.print("-");    Serial.print(now.day(), DEC);       Serial.print("   ");
  Serial.print (now.hour(), DEC); Serial.print(":"); Serial.print(now.minute(), DEC);   Serial.print(":");  Serial.println (now.second(), DEC);
  Serial.print ("Tijd: "); Serial.print(millis()-start); Serial.print("\n");
  
  //Synchronize the RTC with NTP
  Synchronize_RTC();
  delay(50);
  now = RTC.now();
  Serial.print ("Corrected time with NTP: ");
  Serial.print(now.year());     Serial.print("-"); Serial.print(now.month(),DEC); Serial.print("-");  Serial.print(now.day(), DEC);       Serial.print("   ");
  Serial.print (now.hour(), DEC); Serial.print(":"); Serial.print(now.minute(), DEC);   Serial.print(":");  Serial.println (now.second(), DEC);
  delay (5000);
  Serial.print ("Tijd: "); Serial.print(millis()-start); Serial.print("\n"); Serial.print("\n"); Serial.print("\n");
}

//==========================================================================
unsigned long sendNTPpacket(IPAddress& address)
// send an NTP request to the time server at the given address, set all bytes in the buffer to 0
  {
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  // Initialize values needed to form NTP request (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;            // Stratum, or type of clock
  packetBuffer[2] = 6;            // Polling Interval
  packetBuffer[3] = 0xEC;         // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12]  = 49;
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;

  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp: 		   
  Udp.beginPacket(address, 123); //NTP requests are to port 123
  Udp.write(packetBuffer ,NTP_PACKET_SIZE);
  Udp.endPacket();
  }  //End sendNTPpacket

//============================================================================
void Synchronize_RTC()
  {
  sendNTPpacket(timeServer); // send an NTP packet to a time server

  // wait to see if a reply is available
  delay(1000);
  if ( Udp.parsePacket() ) { 
      // We've received a packet, read the data from it
      Udp.read(packetBuffer,NTP_PACKET_SIZE);  // read the packet into the buffer
      //the timestamp starts at byte 40 of the received packet and is four bytes, or two words, long. First, esxtract the two words:

     // NTP contains four timestamps with an integer part and a fraction part. We only use the integer part here
      unsigned long t1, t2, t3, t4;
      t1 = t2 = t3 = t4 = 0;
      for (int i=0; i< 4; i++)  {
          t1 = t1 << 8 | packetBuffer[16+i];     
          t2 = t2 << 8 | packetBuffer[24+i];     
          t3 = t3 << 8 | packetBuffer[32+i];     
          t4 = t4 << 8 | packetBuffer[40+i];}
      //Serial.print(t1); Serial.print("   "); Serial.print(t2); Serial.print("   ");Serial.print(t3); Serial.print("   ");Serial.println(t4);
     
      // part of the fractional part, could be 4 bytes but this is more precise than the 1307 RTC which has a precision of ONE second
      // in fact one byte is sufficient for 1307
      float f1,f2,f3,f4;
      f1 = ((long)packetBuffer[20] * 256 + packetBuffer[21]) / 65536.0;     
      f2 = ((long)packetBuffer[28] * 256 + packetBuffer[29]) / 65536.0;     
      f3 = ((long)packetBuffer[36] * 256 + packetBuffer[37]) / 65536.0;     
      f4 = ((long)packetBuffer[44] * 256 + packetBuffer[45]) / 65536.0;

      //const unsigned long seventyYears = 2208988800UL;
      const unsigned long HundredYears =3155673220UL;
      t1 -= HundredYears;
      t2 -= HundredYears;
      t3 -= HundredYears;
      t4 -= HundredYears;
     
      // Adjust timezone and DST... in my case substract 4 hours for Chile Time or work in UTC?
      //t4 += (2 * 3600L);     // Notice the L for long calculations!!
      t4 += 1;               // adjust the delay(1000) at begin of loop!
      if (f4 > 0.4) t4++;    // adjust fractional part, see above
      RTC.adjust(DateTime(t4));
  }  //End ParsePcket
}   //End function Synchronize_RTC

That value for HundredYears can't be correct. There are 86400 seconds in a day, so HundredYears must end with at least two zeros.
Try:

const unsigned long HundredYears = 3155760000;

Pete

Hallo Pete,

Apart from the right value of the days within 100 years (you are right, it should end with 00), I have a code-problem: the code leads sometimes to fluctuating results.

Sometimes instructions will not be executed.

You are going to have to copy and paste the text from your serial monitor into a code block here so that I can see it.

Pete

Hallo Pete

Enclosed my latest ino-file and a printout of the serial monitor.

Within a loop:

  • synchronizing the RTC with the PC-time
  • Manually setting of a wrong time
  • Correcting the RTC time with NTP.

Problems (see enclosed picture):

  • In the first run the NTP-correction is not executed
  • In the tirth run the NTP-correction is not executed
  • In the fourth run the synchronizing with the PC failed;
  • In the fifth run the synchronizing with the PC failed again.

NTP_04.ino (7.38 KB)

ArduinoStarter1:
I tried to find out the number of seconds for 100 years for adjusting my RTC with NTP.
In the serial monitor ou can see that the time difference between the RTC-time (set with RTC.adjust(DateTime(DATE, TIME)):wink: and the NTP is not stable.

RTC.adjust(DateTime(__DATE__, __TIME__));

Are you aware that this line sets the time to the time of compilation and upload using the DATE and TIME macros?

It is not getting the date and time real-time in your loop() function.

void Synchronize_RTC()
  {
  sendNTPpacket(timeServer); // send an NTP packet to a time server

  // wait to see if a reply is available
  delay(1000);
  if ( Udp.parsePacket() ) {

Is there any guarantee that that the time server will have responded within about one second? If not, you will have waited for about one second in Synchronize_RTC and then returned without having done the NTP correction. The time you print will therefore be about one second later than the "wrong" one.
I say "about one second" because there's no guarantee that delay(1000) is exactly one second. It could be many milliseconds more or less than one second. If it is less than one second it could explain why the NTP update doesn't always take place. Try increasing the delay to, say 1020 and see what happens.
Using delay isn't the best way to do this.
You might be better off removing the delay altogether and wait until parsePacket() returns true - you're not doing anything else at that time anyway.

Pete

@BulldogLowell - using RTC.adjust(DateTime(DATE, TIME)) is deliberate. The clock is set to a known invalid time and is then later on updated with the (hopefully) correct time from an NTP server.

Pete

el_supremo:
@BulldogLowell - using RTC.adjust(DateTime(DATE, TIME)) is deliberate. The clock is set to a known invalid time and is then later on updated with the (hopefully) correct time from an NTP server.

Pete

I can read the code, yes...

His problem (it seemed to me) that is the times went out of sync...

His example shows his NTP time getting to the correct time (I'm assuming by the time and date posted).

el_supremo:

void Synchronize_RTC()

{
  sendNTPpacket(timeServer); // send an NTP packet to a time server

// wait to see if a reply is available
  delay(1000);
  if ( Udp.parsePacket() ) {




Is there any guarantee that that the time server will have responded within about one second? If not, you will have waited for about one second in Synchronize_RTC and then returned without having done the NTP correction. The time you print will therefore be about one second later than the "wrong" one.
I say "about one second" because there's no guarantee that delay(1000) is exactly one second. It could be many milliseconds more or less than one second. If it is less than one second it could explain why the NTP update doesn't always take place. Try increasing the delay to, say 1020 and see what happens.
Using delay isn't the best way to do this.
You might be better off removing the delay altogether and wait until parsePacket() returns true - you're not doing anything else at that time anyway.

Pete

+1. You should not be delaying at all.

BulldogLowell Thanks for the info that RTC.adjust(DateTime(DATE, TIME)); only sets the RTC-time after compiling.

I modified the code without delays (except delay(5000) because it is forbidden to send a NTPpacket too frequently).

I seemed that my function "Synchronize_RTC()" responds most times correct; I decided to repeat my function if it fails. With an DS3231 I can synchronize my timer at midnight.

I will switch over from DS1307-board to DS3231.
With an integrated crystal the inaccuracy of DS3231 is max 3,5 ppm (110 sec/year).
The inaccuracy of a DS1307 is fixed by the quality of the external crystal, but can go up to 45 ppm (more than 1400 seconds per year).

Enclosed my latest program

NTP_05.ino (7.61 KB)

ArduinoStarter1:
I modified the code without delays (except delay(5000) because it is forbidden to send a NTPpacket too frequently).

That is not a good reason. I have working code that queries NTP every 20 seconds. I do not use delay().

Hallo Aarg,

Can you show us your code for synchronizing RTC, you make me curious.