Hello, I need to set the RTC clock on my SAMD21 processor on startup. I'm using an MKR NB 1500 to do an NTP request to a local server, but the UNIX time given is UTC, I need to convert it to GMT+1. I have seen simple formulas where you use the modulo operator (%) to convert the UNIX time into years, months ... , minutes, and seconds. But this gives me the current date and time according to UTC. Could I just add 3600 seconds (1 hour) to the UNIX time and then set my RTC with this time-zone adjusted time? Or would the time start to be inaccurate over a longer period of time because of some time-adjustment phenomena? The RTC clock will only be set again if the watchdog time runs out and the circuit is reset.
Can't you use the time library? I think this will take care of these problems for you.
Yes, but do you want to take account of daylight saving time changes or the equivalent in your country ?
Have you looked at the functions available in the Time library to do conversions between formats amongst may other things ?
Do you mean the <TimeLib.h> library? Will daylight time savings make a big difference over a longer time? The time only needs to be within a minute accuracy.
Yes, that is the library that I had in mind
As to daylight savings time, the clocks change by 1 hour twice a year in Norway
Are there some tricks I can do to adjust accordingly to this?
If you know the rules for when the changes occur, then yes
Do they always go forward by 1 hour at 02:00 on the third Sunday of March and back by one hour at 02:00 on the third Sunday of October, for instance ?
Note that when the clocks go back some times will occur twice in the same day which can make a mess of timed events if they are scheduled with the one hour period and when they go forward some times will not occur on that day
Programming for times can get complicated
1 Like
I've had success with the JChristensen Timezone Library.
1 Like
What core are you using?
Does it include the gnu time libraries which have all the timezone support built in?
yes I have seen implementations that attempt do offset hacks, but that is not the way time tracking is supposed to work and it breaks certain uses of the time.
Over the years I have seen and conversed with MANY people and developers that do not understand how epoch time tracking works.
This misunderstanding has even creeped into many libraries over the years.
This even includes libraries like Arduino TimeZone, Arduino EzTime, and some of the ESPxx API time functions that attempt to insert a timezone offset are not handling time correctly.
Some of the issues in Arduino are due to code using the Time/TimeLib library which is not timezone aware and using hacks to trick it into producing local time looking values.
While they may produce an output that represents the local time they are destroying the actual time_t value and what it represents which may or may not work for some applications.
The time_t value (the seconds since jan 1, 1970) is a fixed point in time.
i.e. the time_t value at any instance in time is supposed to be the same identical value everywhere on earth.
The time_t value is never altered/modified based on the local time.
This is a very important concept to understand.
A time_t value is a timestamp. The time_t value tells you exactly when something occurred but does not tell when that was in local time.
In other words the time_t value is the way computers track time because it is very efficient and low overhead way to track time and local time is just something that humans want to see so the conversion from time_t to a local time is only done when needed since it is a somewhat expensive conversion.
So the time_t value is converted to a local time whenever a human wants to know when that point in time was relative to where the human is positioned on the earch (i.e. the point in time in the local time zone)
This methodology also allows for things like daylight / summer time conversion since that just becomes a rule in the time_t to local time conversion and does not affect the actual time_t value.
i.e you can convert the same time_t value to different local times depending on the local time conversion rules.
While you can munge up the time_t value to trick non time-zone aware time conversion code into producing what looks like a local time from a time_t value you have then destroyed the ability to truly know when something occurred.
While there are applications where this might be acceptable such as a simple clock using non time-zone aware code that thinks it is displaying UTC, it doesn't work for other applications that need to really know when something actually occurred.
The Arduino Time/TimeLib is an example of a library that is not time zone aware.
I had conversations with Michael Margolis about this.
The library was initially a port of some of the unix time library functions but was simplified to reduce its size for the early Arduino processors with limited code and RAM space.
One of the reasons Michael never added the timezone support it is, wait for it....
He lives just outside London. So he luckily never needed timezones.
And while you can get away with modifying the time_t value to trick Time/TimeLib into producing values that look like a local time, it is modifying the actual time.
This may or may not work depending on your applications.
Other Arduino libraries, like TimeZone use an offset hack to produce a local time since they are using Time/TimeLib
And the ESPxx core has some API functions to set an offset from UTC that can be used but they will break the included gnu time library functions so those offset functions should be avoided and the gnu time library functions should be used instead.
The gnu time library functions have built in support for time zones by setting a POSIX timezone rules string. This properly handles the conversions and does not alter the time_t values.
--- bill
2 Likes
On ESP32, is this the correct "GNU Way"?
configTime(gmtOffsetSec, daylightOffsetSec, ntpServer);
Why is that a reason that he never needed timezones ?
Actually that sounds very interesting. Could come in handy even if you can only jump one way.
a7
No.
Using the offsets will cause all the standard time API functions to misbehave.
i.e. localtime() and many other functions will no longer work correctly.
Even worse depending on which API function you use, you will get different "local" times. The hour might be UTC, it might be ahead by the offset, and might be behind by the offset. It is a wreck.
If you want your time API functions to work correctly, never ever use that offset stuff.
The ESP32 is similar to the ESP8266 stuff in the way the offset stuff was implemented.
It is implemented in the wrong layer and the wrong way and even worse it was totally unnecessary.
Basically it was a bad idea and implemented by somebody that didn't understand how the time_t value works and what was already provided by the standard time library functions.
I had conversations with the ESP82xx developers about it.
They agreed but it was broken but didn't want to remove it for fear of breaking something like some existing code that depended on it.
(any existing code that uses it is pretty much broken and likely to have issues)
But they did agree to some fixes I provided to fix the examples to use the newer API functions that use the TZ string.
On the down side, the ESP32 guys are not so easy to work with.
They basically snubbed me and didn't want to talk about this or some other related API issues. They also didn't want to update their examples to show how to properly use timezones.
There are also some additional issues in the ESP32 core APIs related to this.
The ESP32 developers working on this stuff seem to not understand C++ overloading. Instead of providing some overloads for configTime() the created new function names. This makes the API between the ESP32 and the ESP8266 inconsistent. The ESP32 guys did not want to even discuss this issue even though it is a trivial fix for them to resolve this API difference.
Also, the ESP32 core API functions don't include support for a callback that gets called when the NTP background process sets/updates the time.
This is useful as it can be used to set the time inside an RTC if one is being used.
You want to use the APIs that set and use the TZ string.
There are two ways to do this.
One is to use 0 (zero) offsets in configTime() and use the setenv() and tzet() functions to set the TZ string and other is to use the configTime() call that takes a TZstring parameter.
Here is a comment from one of my examples that uses configTime()
// DO NOT attempt to use the configTime() that uses timezone offsets !!!!!
// configTime(tzoffset,dstflg, "ntp-server1", "ntp-server2", "ntp-server3");
// The timezone offset code is really broken.
// if used, then localtime() and gmtime() won't work correctly.
// set both to zero and get real timezone & DST support by using TZ string
// enable NTP by setting NTP server
// up to 3 ntp servers can be specified
// configTime(0, 0, "server1", "server2", "server3");
// There is a configTime() that supports a TZ string
// enable NTP by setting up NTP server(s)
// up to 3 ntp servers can be specified
// configTime(TZstr, "server1", "server2", "server3");
// If not using a network, or NTP, don't use configTime()
// NOTE:
// Unfortuntely it is called configTzTime() on esp32
// configTzTime(TZstr, "server1", "server2", "server3");
That said, I had an example I was going to post and just just tried it to make sure it still works.
While it works on the esp8266, I just tried the configTzTime() option with the TZstring parameter on the ESP32 and apparently it no longer works (it crashes the processor).
So it may be best to use the setenv() and tzset() calls with the zero offsets.
Here is link to a page that has a good example with some explanation about using setenv() and tzset() to setup up the timezone information.
again, never ever use the offset stuff and use the TZstring as that is what the time libraries use to calculate the local time information.
If you use the TZstring all the time function APIs will "just work" and work as documented and expected.
--- bill
1 Like
Think about it. He is in London. London is GMT/UTC
Does UTC change twice a year when the time in London changes ?
I don't follow.
Unless by "jump one way" you mean alter the time_t to a local time for a specific purpose knowing you can never get back to the original time_t.
yes that can work and work well for some simple environments that are very localized like a simple clock.
But If you modify the actual time_t value you then lose when the event actually occurred.
If you use the time_t as a timestamp, then when looking at just the timestamp you now have no clue when something actually happened since the timestamp no longer represents an offset from the true epoch but has been modified to some other point in time that is not reflected by nor can be derived by simply having the timestamp.
You need a magic secret decoder ring to be able to know the point in time that the timestamp repents to be able to shift it back to the epoch offset.
This becomes unworkable if you have things in multiple locations that are using local time or munged time_t values.
When time_t represents an offset from the epoch, then the time_t is self explanatory as to representing an exact point in time.
i.e. If all you have is a local datetime or a modify time_t value then you need to know additional information to be able to get to the original time_t to be able to get to the exact point in time.
Same issue if all you have is a local datetime.
You have to know how the local date / time value was adjusted from the UTC to be able to figure out what point in time is being represented.
Now for some simple applications knowing the exact point in time is not necessary. i.e. a simple clock that always represents the local date / time.
But for for other applications, this is not workable.
IMO, I prefer to stick to the POSIX time functions that handle all the timezone stuff through using TZ rules that can also do the automatic daylight savings adjustments.
That way the code written works everywhere that supports the standard time calls which have not changed in decades.
Have I ever done it the cheating way by hacking the time_t values to track the incorrect local time rather than the true UTC time? Absolutely.
Most recently it was for longevity reasons.
Recently I created a Word clock that I wanted to be very automatic, very reliable, and work for many years as it is for my 80+ yo mother.
It uses multiple methods to get the time, supports an RTC as well as getting the local time from a time server which use IP geographic data get the local time.
I.e. you plug it in and it "just works" to set it self.
It uses an RTC that gets set to the local time from a network server every day at 02:00.15 (30 seconds past 2am)
It uses an esp8266. I started off using NTP - which is when I ran into all the configTime() offset issues.
But eventually I decided to not use NTP and to use the Arduino Time/TimeLib library with a hacked/munged time_t value to trick it to provide local time values.
The reason I use Time/TimeLib is that at this point in time, pretty much all the time libraries in the Arduino microcontroller cores will fail in 2038 due to 32 bit rollover. (all but the STM cores are currently using a 32 bit time_t)
The Time/TimeLib uses a 32 bit unsigned int to allow it to reach up to about 2106
It sacrifices pre 1970 dates but I don't need those so it is ok for me.
I wanted a clock that would work up to and beyond 2038 - which is not all that far out now.
The 2038 problem is a MUCH bigger problem than y2k.
--- bill
neither GMT nor UTC do time offsets.
I guess I was wrong about London time.
London doesn't use GMT. (well it is the same as GMT in the winter)
London does summer time offsets so it will be off from GMT during the summer.
--- bill
That is what I thought, so why
It looks to me like that's what the code I posted does. Tracing through the library ....
This is configTime():
void configTime(long gmtOffset_sec, int daylightOffset_sec, const char* server1, const char* server2, const char* server3)
{
//tcpip_adapter_init(); // Should not hurt anything if already inited
esp_netif_init();
if(sntp_enabled()){
sntp_stop();
}
sntp_setoperatingmode(SNTP_OPMODE_POLL);
sntp_setservername(0, (char*)server1);
sntp_setservername(1, (char*)server2);
sntp_setservername(2, (char*)server3);
sntp_init();
setTimeZone(-gmtOffset_sec, daylightOffset_sec);
}
The last line calls setTimeZone():
static void setTimeZone(long offset, int daylight)
{
char cst[17] = {0};
char cdt[17] = "DST";
char tz[33] = {0};
if(offset % 3600){
sprintf(cst, "UTC%ld:%02u:%02u", offset / 3600, abs((offset % 3600) / 60), abs(offset % 60));
} else {
sprintf(cst, "UTC%ld", offset / 3600);
}
if(daylight != 3600){
long tz_dst = offset - daylight;
if(tz_dst % 3600){
sprintf(cdt, "DST%ld:%02u:%02u", tz_dst / 3600, abs((tz_dst % 3600) / 60), abs(tz_dst % 60));
} else {
sprintf(cdt, "DST%ld", tz_dst / 3600);
}
}
sprintf(tz, "%s%s", cst, cdt);
setenv("TZ", tz, 1);
tzset();
}
Yes, in the previous post I admitted I was wrong.
He would need to have timezone offset for summer time.
I did talk with Michael on the phone quite a few times and joked with him about not needing time zone support given him living in London area.
He laughed. I just never went back and dug deep enough to see that England does summer time adjustments.
So not sure why he never addressed timezones other than it would make the code larger and for many simple purposes like a click you can cheat using a munged up time_t value. But even that doesn't address automatic DST changes.
But Jack's TimeZone library can do that munging for you if it works for your application.
--- bill