Timestamping And Psuedo Clocks SOLVED!

Projects Programming
1

I have a sketch that retrieves an NTP timestamp off the web as a reference for event logging
in a SPIFFS file on a Wemos D1 mini 8266. The events logged are power outages at my home.

The issue I have is that when power is restored the router takes a couple minutes to initialize and reconnect to the web before the 8266 can get another timestamp to use to record the
power restoration time. I was trying to avoid using a RTC module since timestamping is available. Is there code out there that mimics a real time clock and could be updated by the latest timestamp. Even if off a few seconds that is not critical to my logging needs. But being off a couple minutes is not welcomed.
So before I go with an RTC module I wanted to see if anyone had any information about
this type of approach.
P.S. I have thought about a small UPS for the router but that is a more expensive solution.

2

Does the Wemos D1 mini lose power during the power outage? The ESP8266 has an internal RTC, although it is only as accurate as the clock source. Not sure about the ESP8266, I see some comments that it is an RTC emulation, but if the ESP8266 has power during the outage that should still be good enough.

3

In principle, if the ESP has an independent power supply, you can use the Arduino time library to maintain the time. Every so often you fetch the current time using NTP and update/correct the Arduino's system time. You use this time instead of fetching a time stamp for each log entry. The longer the intervals between updates from NTP, the more risk there is of some drift in the time.

4

Negative. It has battery backup.

I was not aware of any RTC emulation. I need to investigate this.

5

Bingo!!!! Was not aware of the Arduino Time Library!!!
Thank you. Hopefully I can find something there that will meet my needs. :+1:

edit - The Time Library seems to be primarily for getting web timestamps and using this in conjunction with an RTC.

6

be careful, when people say "Arduino time" library likely they mean they Arduino Time/TimeLib library.

I would use the standard time functions that come with the ESP8266.
(no Arduino library needed and you get NTP synchronization and timezone & automatic DST offset support)
It provides all the standard time functions time(), timeofday(), gettimeofday(), localtime(), etc... i.e. the standard time function APIs that have existed for 50 years.

The ESP8266 will run NTP synchronization in the background for you and you can configure your local timezone if you want to output the local time including DST adjustments.

It is very simpe to configure as you just configure the TZ string and NTP server(s)
The OS on the ESP8266, keeps time in the background and every few minutes syncs with an NTP server.
If the network goes down, then time is still kept but the accuracy can start to drift a little bit depending on the crystal accuracy and how long the network is down.
Once network connectivity is restored, it will start syncing with an NTP server again.

There is no need for an RTC.

--- bill

1 Like
7

Bill, can you post a link to the documentation for this please? When I google for it, all the articles I find involve installing an NTPClient library (from IDE library manager). But you are saying there is something similar that is installed with the ESP8266 boards package?

8

You can follow the links in this video to see an simple example. It may not be completely up to date: #299 Tricks to get NTP time for the ESP32 and the ESP8266 incl. Summer- and Daylight Saving time - YouTube
In the simplest case, it is only a few lines of code to use the built in functionality of the core.

EDIT

And here is the example from the ESP8266 core: Arduino/NTP-TZ-DST.ino at master · esp8266/Arduino · GitHub
It is quite cluttered but the essential part of it is the configTime() function.

9

I am interested to know more about this. If the 8266 can be given a timestamp and then go with that to keep time independently in the background.

Extreme accuracy is not an issue in my application. But so far I have been unable to put my finger on how to access and retrieve any time data from the processor's OS.
Any leads are welcomed.

10

This piqued my interest as well. If I can request time data from the processor OS that would make using an RTC module unnecessary.

11

Yes.
The esp8266 includes full NTP and time library support.
These are time library functions that were defined by unix over 50 years ago, and are still used today. Many entities, including Microsoft chose to ignore these APIs but slowly after a few decades came around and implemented most of them.
Today the time library API functions are included in the gnu time library.
You can see the standard gnu library API functions by doing something like:
google ctime
If you use an OS that is for s/w development like linux you can simply
man ctime

Many Arduino cores do come with at least partial gnu time library support.
Do not confuse this with the Michaeal Margolis's Time/TimeLib library.
Michael did that library years ago as a way to provide time library like services to Arduino. The API for the Arduino Time (now TimeLib) library was based on the standard unix time library APIs but was modified to make it smaller for AVRs with limited code space and to make it more "Arduino like" including using cutesy camelcase names which people like Massimo and Igoe were pushing for.

There is an example that comes with the ESP8266 core here:
File->Examples->ESP8266->NTP-TZ-DST

An issue in the ESP826 core is that the way the implemented the offset stuff in the low level NTP and gnu library glue is totally broken and breaks all gnu time library functions that do all the local time calculations.
They implemented some things at the wrong level and incorrectly.
So NEVER EVER use any of the offset stuff. Use the API functions that use the TZ string.
The TZ stuff is better and provides automatic DST changes, so there really is no need to ever use the offset stuff.

In terms of documentation, I haven't found anything formal from the ESP8266 guys that describes the NTP and low level glue functions like configTime()
I went by the actual code. They provided me access to their repository years ago as I was working with them on some issues in the time code.

They have read the docs here which is a fantastic source of ESP8266 information:
https://arduino-esp8266.readthedocs.io/en/latest/
But it doesn't seem to have anything for the low level time functions that are the glue for the gnu time library and the NTP code.
Here is a link to one of the main module:

Again, use the functions like configTime() that use the TZ string, never ever use the offset stuff. If you do, you will be sorry since it mucks with the time_t values which breaks many things like timestamps and the local time calculations for the gnu time library.

Here is a github issue that I created a few years ago that can provide some context and history as to some of the issues in the API. Some of the issues have been worked around but some like the timezone offset in the configTime() still remain and can't/won't ever be fixed.

If you want more I can post an example that I created that has comments that describes some of uses and "gotchyas" that are not mentioned in the ESP8266 NTP example including some of the still remaining issues in the code.

--- bill

2 Likes
12

Yes you can.
An RTC is unnecessary if you have WiFi connectivity to the internet.
I have several ESP8266 based clocks that keep accurate time and have no RTC.

You just configure the system for the timezone using a TZ string and point it to the NTP servers and boom, you have accurate time starting in a second or so once it does that first NTP sync.

You can use a callback to tell when a sync happens.

I have an ESP8266 based clock that disables WiFi but powers it back up to let it sync to NTP once per day. During the day it is fee running with no NTP syncs.
The time stays within better than 1 second accuracy.

One big issue that is starting to affect clock implementations is the length of time_t values.
32 bit values will break in 2038.
This is a MUCH bigger issue that y2k ever was.

There are various work arounds that can buy some time, but basically implementations need to start moving to 64 bit time_t values.

--- bill

1 Like
13

Thanks Bill, will absorb all this later. I have a new clock project to work on, plus a couple of others that I could retrofit with these functions when I have a reason to change them.

14

This is encouraging. Can you direct me to an Arduino IDE compatible code source that shows how to set/initiate an internal time request such as:

rtctime.set()
rtctime.get()

rtctime.set() : Sets the rtctime to a given timestamp.

source: rtctime - NodeMCU Documentation

I am still unfamiliar with how to set an 8266 internal clock once I receive a web NTP timestamp. This is a huge hurdle for me currently. If I can figure this out I can then do a once per day NTP request and keep the processor fairly accurate.

edit: after user gfvalvo pointed out that library sketch:NTP-TZ-DST also sets up the processor "RTC" I hope to be able to use this code to retrieve current time. :+1:

15

Did you see this:

image
image723×970 31 KB

16

Thank you. I thought, wrongly, by the title it was only about NTP timsestamping but it seems to
offer what I need just looking at it quickly. :+1:

17

That isn't how it works.
You don't mess with NTP in the sketch. The OS on the ESP8266 handles it all in the background just like it does in on other bigger other OS platforms like linux, MacOS, or Windows.
All you do is call configTime() set the TZ string for the local time zone information and point it to the NTP server(s).
You then use the standard time API funcitons to access the time.
including time() to get the time_t value.
If WiFi is enabled & active then the background process will attempt to sync every so often. (I think it is like every 15-20 minutes).

Remember time_t values are the same everywhere on the planet at any given point in time and is not affected by any local timezone rules.
That is what makes it ideal for a timestamp.

--- bill

1 Like
19

Ok folks, my learning curve is straightening out a tad, finally.

I found this bit of code over at the ESP8266 Community Forum:

type o#include <ESP8266WiFi.h>
#include <time.h>

const char* ssid = "..........";
const char* password = "..........";

int timezone = 3;
int dst = 0;

void setup() {
  Serial.begin(115200);
  Serial.setDebugOutput(true);

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.println("\nConnecting to WiFi");
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print(".");
    delay(1000);
  }

  configTime(3 * 3600, 0, "pool.ntp.org", "time.nist.gov");
  Serial.println("\nWaiting for time");
  while (!time(nullptr)) {
    Serial.print(".");
    delay(1000);
  }
  Serial.println("");
}

void loop() {
  time_t now = time(nullptr);
  Serial.println(ctime(&now));
  delay(1000);
}

https://www.esp8266.com/viewtopic.php?p=73494#

It is short and sweet and I can easily adapt this to my sketch. I can now hopefully get out of your collective "hair" and move on with my project.

thanks for everyone's input... :+1:

20

That code uses the "offset stuff" that @bperrybap warned you about. From the offsets you put in, it looks like you're located either somewhere 3 hours east of UTC that doesn't observe DST or somewhere 2 hours east of UTC that's currently observing DST. You can't tell, and I guess that's one of @bperrybap's peeves about the "offset stuff".

So, try this ... pick your time zone from tz.h and use the other overload of configTime(). Let's assume the latter of my above two possibilities. So:

   configTime("EET-2EEST,M3.5.0/3,M10.5.0/4", "pool.ntp.org", "time.nist.gov");

Or, use the human-readable forms that tz.h provides for you. Then something like:

   configTime(TZ_Europe_Bucharest, "pool.ntp.org", "time.nist.gov");
21

Understood. I had not yet corrected the international time zone parameter when I posted that code. For CST it requires a "-5". Admittedly tz.h is a big improvement.