Simplest way of timekeeping?

I've done some tests with millis(), but it seems there's a noticeable drift after only a couple of hours

Could be your code is at fault.
But sadly, we can't see it.