There is no way to keep accurate time the way you doing it.
You should take a look at using the Time library.
It will keep more accurate time since it keeps time in the background using the millis() timer.
But even if you use the Time library, you can expect some drift since the crystal on an Arduino is not very accurate.
If you were using a esp8266 based module, you could use the WiFi connection to sync with an NTP server and have very accurate time keeping.
--- bill