My way to keep millis() accurate

Community Exhibition / Gallery
#1

I have been building an electronic system for pyrotechnic firing, and one of the things I knew I needed was accuracy to about 1/100s maintained for up to 20 minutes. I hoped at first that the Arduino’s clocks would be good for this level of accuracy. However, when I first transmitted a firing script from my Mega2560 based Controller to my first Uno based Field Module, and told them to go, the module’s idea of 1ms was significantly different from the Mega’s. When I measured it, they were drifting apart at the rate of about 10s/hour.

Although I didn’t know which was the truest, my first objective was to keep the two in sync. I decided that a reasonable approach was to broadcast a message from the Controller to all Field Modules every 10 seconds, containing nothing more than the Controller’s current millis() value. Each time the module received the value, it would store it, and it’s own current millis() value, compare these with the previous value (10s ago) and compute a scale factor required to convert its millis() value to that the Controller sent. I then wrote myMillis() which took millis() and multiplied it by the scale factor. I used myMillis() within my own code, and the two clocks now keep the same time. Only it probably isn’t good time.

I splashed out on a DS3231 RTC as I reckoned that this would solve my problems. It is temperature compensated, which is handy on a system that may operate at +30C in summer, or -10C in winter. I hadn’t read all the useful stuff on this forum before I bought it, so was a little surprised to find that despite its apparent accuracy, it can’t provide anything more granular than 1 second. That’s not much good, I thought. Then I found a couple of posts that suggested using the RTC’s SQW signal. One post suggested using this to interrupt the microcontroller. Unfortunately, as a software developer and not an electronic engineer, I found the details less than explicit, and one of the reasons for posting this is so the next person wondering how to do this can benefit from my experimentation.

My functional design was to use the RTC’s SQW output selected at a 1 second frequency, and use the same approach to synchronise the Controller’s Mega to the RTC as I had to synchronise the Field Module to the Controller. At each interrupt, the service routine would note the current value of millis() and compare this with the value it stored last second. Anything above or below 1,000 represents a drift and these drifts are added together. Another version of myMillis() this time takes the current value of millis() and subtracts the cumulative drift. Consequently, any 2 calls to myMillis() will result in a value which is very close to the actual number of milliseconds that have elapsed.

That’s the design. This is what I did.

My DS3231 RTC is on a ZS-042 board. This has 6 pins, of which I used only GND, VCC, SDA, SCL & SQW.

GND is connected directly to GND on the Mega.
VCC is connected to 3V3 on the Mega.
for these two, I used the block near the power jack.
SDA is connected to SDA on the Mega.
SCL is connected to SCL on the Mega.
For these two, I used the connectors near the USB connector.
SQW is connected to pin 18 on the Mega. Additionally, I have a 4k7 Ohm resistor between pin 18 and 5V on the Mega.

By way of explanation of the choices here, I have a CTE 5" LCD on its shield sitting on the Mega, and also have an XBee shield on flying wires soldered onto other pins, so have to use what is available and most convenient.

To test this that this connection functioned, I took some code that has already been posted here, modified it a bit and ran it. It appeared to work, in that I could set the time and retrieve it.
This was on a day when my office was getting seriously hot so I printed the drift value every 10 minutes and plotted this against the temperature (that the DS3231 supplies) on a graph. The correlation between temperature and drift on the Mega’s clock is surprisingly close and shown in the attached file.

I left the RTC running for a couple of weeks, comparing it regularly with my radio adjusted watch, and I have to say that it hasn’t shifted. On the other hand, the drift of the Mega’s clock is accumulating at a few seconds (gain) each day, so I am quite happy that my fireworks will go off very near to the 1/100s tick that they are meant to.

I’ll wrap this up by offering the code I used to play with the DS3231. It’s not particularly well commented, but it does show how you can keep track of elapsed times that are close to millisecond accuracy over protracted periods.

Mega Clock vs Temp.png
Mega Clock vs Temp.png1775×856 72 KB

1 Like
#2

The code for the above post - part 1.

#include <Wire.h>

// Not sure if the I2C address has any meaning when using SDA/SCL
#define DS3231_I2C_ADDRESS 104

// The possible frequencies for the DS3231
#define DS3231_SQW_FREQ_1    0b00000000 // 1Hz
#define DS3231_SQW_FREQ_1024 0b00001000 // 1024Hz
#define DS3231_SQW_FREQ_4096 0b00010000 // 4096Hz
#define DS3231_SQW_FREQ_8192 0b00011000 // 8192Hz

// The frequency for the serial output
#define N_MINS 1

// Constants for the connections. Change this if using a different board
// or different pins on a Mega2560
#define INT_NUM 5
#define INT_PIN 18
#define LED_PIN 13

// We will be talking directly to the millis() counter
extern unsigned long timer0_millis;

//Structures for manipulating the comms with the DS3231. Probably overkill.
struct Temperature {
  int8_t tempDeg;
  uint8_t tempFrac;
};

union dsTemperature {
  struct Temperature tempPart;
  int16_t tempAll;
};

typedef struct ds3231dtt_s {
  union dsTemperature temp;
  byte seconds, minutes, hours, day, date, month, year;
} ds3231dtt_t;

// This is something of a waste of static space
char *dayNames[] = {"Sun","Mon","Tue","Wed","Thu","Fri","Sat"};

// Volatile storage for anything that will be changed by the ISR
volatile unsigned long lastmillis=0;
volatile long drift=0;
volatile boolean trigger1s = false;

// We will flash the LED at exactly 1s intervals
boolean ledOn = false;

void setup() {
  // Standard initialisations
  Wire.begin();
  Serial.begin(9600);
  // 1 second inputs please
  setSQWFrequency(DS3231_SQW_FREQ_1);
  SQWEnable(true);
  
  pinMode(INT_PIN,INPUT);
  pinMode(LED_PIN,OUTPUT);

  // Our interrupt handler added to INT_NUM
  attachInterrupt(INT_NUM, TIME_ISR, FALLING);

  // Start by getting the current time and printing it
  Serial.print("START @");
  ds3231dtt_t ds3231dtt;
  get3231TimeDateTemp(&ds3231dtt,false,false);
  SerialPrintTimeDateTemp(&ds3231dtt,false,false,true,true);
}

void loop() {
  static byte last_N_Min = 0xFF;
  static int16_t lasttemp = 0;
  ds3231dtt_t ds3231dtt;
  boolean outputRequired = false;

  // If there is anything coming in on Serial then handle it
  if(watchConsole()) {
    // A true return indicates that the clock time has been changed
    // so we re-initialise our counters.
    lastmillis=0;
    drift=0;
    outputRequired = true;
  }

  // trigger1s is set whenever the interrupt arrives. Next time
  // we enter loop(), we will deal with it
  if(trigger1s) {
    trigger1s = 0;
    // Use myMillis() to get the time. It is more accurate
    unsigned long m = myMillis();

    // Toggle the LED
    digitalWrite(LED_PIN,(ledOn = !ledOn));

    // Get and print the time
    get3231TimeDateTemp(&ds3231dtt,false,false);

    // Although we see the trigger every second, we are only
    // going to print the time every N minutes, so have we
    // reached that boundary?
    if((ds3231dtt.seconds == 0) && ((ds3231dtt.minutes / N_MINS) != last_N_Min)) {
      outputRequired=true;
    }

    if(outputRequired) {
      // Now we just need to output the time, as we see it
      last_N_Min = ds3231dtt.minutes / N_MINS;
      lasttemp = ds3231dtt.temp.tempAll;

      SerialPrintTimeDateTemp(&ds3231dtt,false,false,false,false);
      // but for comparison, we will also show what millis() and myMillis() says.
      // Could output drift here instead, but it is the difference between them
      Serial.print(" - millis "); Serial.print(millis(),DEC);Serial.print("/");Serial.println(m,DEC);
    }
  }
}

// Convert normal decimal numbers to binary coded decimal
byte decToBcd(byte val)
{
  return ( (val/10*16) + (val%10) );
}

boolean watchConsole()
{
  if (Serial.available()) {      // Look for char in serial queue and process if found
    char c = Serial.read();

    switch (c) {
      case 'T':
      case 't': // set the date & time
          set3231Date();
          return true;
        break;
      case '?': // output the date & time, to the second
          ds3231dtt_t ds3231dtt;
          get3231TimeDateTemp(&ds3231dtt,true,true);
          SerialPrintTimeDateTemp(&ds3231dtt, true, true, true, true);
        break;
      default: // Swallow unwanted input
        while(Serial.available())
          Serial.read();
    }
  }
  return false;
}

void SerialPrintTimeDateTemp(struct ds3231dtt_s *pds3231dtt, boolean printDate, boolean printTemp, boolean printSecs, boolean printNL)
{
  if(printDate) {
    Serial.print(dayNames[pds3231dtt->day]);
    Serial.print(", "); 
    Serial.print(pds3231dtt->date, DEC); 
    Serial.print("/"); 
    Serial.print(pds3231dtt->month, DEC); 
    Serial.print("/"); 
    Serial.print(pds3231dtt->year, DEC);
    
    Serial.print(" - ");
  }
  
  Serial.print((pds3231dtt->hours<10)?"0":""); 
  Serial.print(pds3231dtt->hours, DEC);  
  Serial.print((pds3231dtt->minutes<10)?":0":":"); 
  Serial.print(pds3231dtt->minutes, DEC);

  if(printSecs) {
    Serial.print((pds3231dtt->seconds<10)?":0":":"); 
    Serial.print(pds3231dtt->seconds, DEC);
  }
  
  if(printTemp) {
    Serial.print(" - ");
     
    Serial.print("Temp: "); 
    Serial.print(pds3231dtt->temp.tempPart.tempDeg);
    Serial.print(".");
    if(pds3231dtt->temp.tempPart.tempFrac == 0)
      Serial.print("00");
    else
      Serial.print(pds3231dtt->temp.tempPart.tempFrac);
  }

  if(printNL)
    Serial.println("");
}

byte getByte()
{
  while (0 == Serial.available());
  return Serial.read();
}
  
void set3231Date()
{ 
//T(sec)(min)(hour)(dayOfWeek)(dayOfMonth)(month)(year)
//T(00-59)(00-59)(00-23)(1-7)(01-31)(01-12)(00-99)
//Example: 02-Feb-09 @ 19:57:11 for the 3rd day of the week -> T1157193020209
  ds3231dtt_t l_ds3231dtt;

  byte l_seconds, l_minutes, l_hours, l_day, l_date, l_month, l_year;

  l_ds3231dtt.seconds = (byte) ((getByte() - 48) * 10 + (getByte() - 48));
  l_ds3231dtt.minutes = (byte) ((getByte() - 48) *10 +  (getByte() - 48));
  l_ds3231dtt.hours   = (byte) ((getByte() - 48) *10 +  (getByte() - 48));
  l_ds3231dtt.day     = (byte) (getByte() - 48);
  l_ds3231dtt.date    = (byte) ((getByte() - 48) *10 +  (getByte() - 48));
  l_ds3231dtt.month   = (byte) ((getByte() - 48) *10 +  (getByte() - 48));
  l_ds3231dtt.year    = (byte) ((getByte() - 48) *10 +  (getByte() - 48));

  Wire.beginTransmission(DS3231_I2C_ADDRESS);
  Wire.write(0x00);
  Wire.write(decToBcd(l_ds3231dtt.seconds));
  Wire.write(decToBcd(l_ds3231dtt.minutes));
  Wire.write(decToBcd(l_ds3231dtt.hours));
  Wire.write(decToBcd(l_ds3231dtt.day));
  Wire.write(decToBcd(l_ds3231dtt.date));
  Wire.write(decToBcd(l_ds3231dtt.month));
  Wire.write(decToBcd(l_ds3231dtt.year));
  Wire.endTransmission();
}
1 Like
#3

Part 2 of the code.

void get3231TimeDateTemp(struct ds3231dtt_s *pds3231dtt, boolean getDate, boolean getTemp)
{
  // send request to receive data starting at register 0
  Wire.beginTransmission(DS3231_I2C_ADDRESS); // 104 is DS3231 device address
  Wire.write(0x00); // start at register 0
  Wire.endTransmission();
  Wire.requestFrom(DS3231_I2C_ADDRESS, getDate?7:3); // request seven or three bytes

  if(Wire.available()) { 
    pds3231dtt->seconds = Wire.read(); // get seconds
    pds3231dtt->minutes = Wire.read(); // get minutes
    pds3231dtt->hours   = Wire.read();   // get hours

    if(getDate) {
      pds3231dtt->day     = Wire.read();
      pds3231dtt->date    = Wire.read();
      pds3231dtt->month   = Wire.read(); //temp month
      pds3231dtt->year    = Wire.read();
    }
           
    pds3231dtt->seconds = (((pds3231dtt->seconds & B11110000)>>4)*10 + (pds3231dtt->seconds & B00001111)); // convert BCD to decimal
    pds3231dtt->minutes = (((pds3231dtt->minutes & B11110000)>>4)*10 + (pds3231dtt->minutes & B00001111)); // convert BCD to decimal
    pds3231dtt->hours   = (((pds3231dtt->hours & B00110000)>>4)*10 + (pds3231dtt->hours & B00001111)); // convert BCD to decimal (assume 24 hour mode)
    if(getDate) {
      pds3231dtt->day     = (pds3231dtt->day & B00000111)-1; // 0-6
      pds3231dtt->date    = (((pds3231dtt->date & B00110000)>>4)*10 + (pds3231dtt->date & B00001111)); // 1-31
      pds3231dtt->month   = (((pds3231dtt->month & B00010000)>>4)*10 + (pds3231dtt->month & B00001111)); //msb7 is century overflow
      pds3231dtt->year    = (((pds3231dtt->year & B11110000)>>4)*10 + (pds3231dtt->year & B00001111));
    }

    if(getTemp) {
      //temp registers (11h-12h) get updated automatically every 64s
      Wire.beginTransmission(DS3231_I2C_ADDRESS);
      Wire.write(0x11);
      Wire.endTransmission();
      Wire.requestFrom(DS3231_I2C_ADDRESS, 2);
      
      if(Wire.available()) {
        int8_t tMSB = Wire.read(); //2's complement int portion
        int8_t tLSB = Wire.read(); //fraction portion
        
        pds3231dtt->temp.tempPart.tempDeg = tMSB; //(tMSB & B01111111); //do 2's math on Tmsb
        if(tMSB < 0)
          tLSB = -((tLSB >> 6) & 0B11);  //only care about bits 7 & 8
        else
          tLSB = (tLSB >>= 6) & 0B11;  //only care about bits 7 & 8
        pds3231dtt->temp.tempPart.tempFrac = ( tLSB * 25 );
      }
    }
  }
}


void SQWEnable(uint8_t enable)
{
  Wire.beginTransmission(DS3231_I2C_ADDRESS);
  Wire.write(0x0E);
  Wire.endTransmission();

  // control register
  Wire.requestFrom(DS3231_I2C_ADDRESS, 1);

  uint8_t creg = Wire.read();

  creg &= ~0b01000000; // Set to 0
  if (enable == true) {
    creg |=  0b01000000; // Enable if required.
    creg &= ~0b00000100; // Clear INTCN bit
  }

  Wire.beginTransmission(DS3231_I2C_ADDRESS);
  Wire.write(0x0E);
  Wire.write(creg);
  Wire.endTransmission();
}

void setSQWFrequency(uint8_t freq)
{
  Wire.beginTransmission(DS3231_I2C_ADDRESS);
  Wire.write(0x0E);
  Wire.endTransmission();

  // control register
  Wire.requestFrom(DS3231_I2C_ADDRESS, 1);

  uint8_t creg = Wire.read();

  creg &= ~0b00011000; // Set to 0
  creg |= freq; // Set freq bits

  Wire.beginTransmission(DS3231_I2C_ADDRESS);
  Wire.write(0x0E);
  Wire.write(creg);
  Wire.endTransmission();
}

void TIME_ISR(void)
{
  // Interrupt Service Routine - keep it short & sweet
  unsigned long m;
  uint8_t oldSREG = SREG;
 
  // disable interrupts while we read timer0_millis or we might get an
  // inconsistent value (e.g. in the middle of a write to timer0_millis)
  cli();
  m = timer0_millis;
  SREG = oldSREG;

  if(lastmillis == 0) {
    // Only going to do this first time into the ISR
    lastmillis=m;
  }
  else {
    drift += (m-lastmillis) - 1000;  // The intervening time should have been 1000ms. All else is drift.
    lastmillis=m;
  }

  trigger1s=true;
}

unsigned long myMillis()
{
  return (millis() - drift);
}
1 Like
#4

SkyDemon:
I have been building an electronic system for pyrotechnic firing, and one of the things I knew I needed was accuracy to about 1/100s maintained for up to 20 minutes....

Interesting. I'm curious about the pyrotechnics -- do you mean fireworks?

I was also a little annoyed to discover that most RTCs require you to feed a signal back into a digital pin in order to get resolution better than 1 second. One exception is the PCF85263A RTC which has a register for hundredths of a second (but it isn't temperature compensated). Another way would be to poll the RTC in a tight loop in order to catch the moment when the second changes.

Also, I think you mean "precision of 1/100 second". You aren't actually setting the RTC to the current time +/- 0.01 seconds, are you?

#5

Some more options:
Soft: A settable calibration factor in the timer library would be nice. Or a workaround based on micros().
Hard: Tweak the 16MHz quartz to the correct frequency.

#6

Is this helpfull?

http://wyolum.com/syncing-arduino-with-gps-time/

#7

jboyton:
Interesting. I’m curious about the pyrotechnics – do you mean fireworks?

Yes - although it can also be used for stage pyro, which aren’t generally called fireworks. Being a certifiable pedant, I use the generic term.

jboyton:
Also, I think you mean “precision of 1/100 second”. You aren’t actually setting the RTC to the current time +/- 0.01 seconds, are you?

Yes, to the first point, thank you, no to the second. I am interested in the precision of the timing between one cue firing and the next. The script may say to fire Cue 1 when the start button is pressed, and to fire Cue 2 at exactly 5.02 seconds later. In a musical display, where the music is triggered also on the start button, that second cue might be on the peak of a particular beat. By the time we get to the end of the display, we may want to hit a beat 13 min 28.63 seconds from the start. This event should still be hit at +/- 0.005s to achieve the precision I am looking for.

Wawa:
Is this helpfull?

http://wyolum.com/syncing-arduino-with-gps-time/

Thank you. I did toy with the idea of GPS, but I have no requirement for the positional functionality, so not to use it seems a bit of a waste, and the DS3231 is significantly less expensive.

#8

DrDiettrich:
Some more options:
Soft: A settable calibration factor in the timer library would be nice. Or a workaround based on micros().
Hard: Tweak the 16MHz quartz to the correct frequency.

It wouldn't have to be the correct frequency, just a stable one. The problem is that SkyDemon needs stability of ±8ppm over a 40°C temperature range. He could use a temperature sensor (or maybe even the internal thermometer of the processor) and adjust for it in his code. But then he'd have to spend time calibrating it. It's easier to use a temperature compensated RTC.

#9

SkyDemon:
I am interested in the precision of the timing between one cue firing and the next. The script may say to fire Cue 1 when the start button is pressed, and to fire Cue 2 at exactly 5.02 seconds later. In a musical display, where the music is triggered also on the start button, that second cue might be on the peak of a particular beat. By the time we get to the end of the display, we may want to hit a beat 13 min 28.63 seconds from the start. This event should still be hit at +/- 0.005s to achieve the precision I am looking for.

Thanks for the explanation of why you needed the timing.

#10

I looked briefly at your code. I hadn't realized that the square wave output of the DS3231 (I have the SPI version of that chip) goes low when the second changes. I hadn't read the datasheet carefully enough and assumed it was a rising edge.

One small thing I noticed was that you are disabling interrupts within your ISR. Interrupts are already disabled automatically so that's unnecessary. It doesn't really matter though. It just makes the ISR a tiny bit longer than it needs to be.

#11

Temperature is a minor problem. There exist heaters which keep the temperature of a component (quartz...) or inside a box sufficiently constant.

I doubt that an overall 0.005s accuracy is required, the uncertainty of a thermal ignition will be much higher. But stability and precision of the time source is important, so that the music and ignitions stay in sync. Best synchronization could be achieved when the sound (MP3...) clock were transmitted, so that also deviations of that clock will be taken into account.

#12

Not sure if you didn't try, but what about running a separate clock/synch line output to by a master and input by your slave units?

You should be able to run over 100 timed tasks with better than 1/100th second (10ms) accuracy without using interrupts at all. Too much interrupt will throw your clocks off.

Are you familiar with BlinkWithoutDelay?

#13

DrDiettrich:
Temperature is a minor problem. There exist heaters which keep the temperature of a component (quartz...) or inside a box sufficiently constant.

Do you have a link to one of these heaters?
Would it really be easier than using a TCXO?

#14

An OCXO (oven controlled...) works in a wide range of ambient temperatures, in detail in cold environment. I like that idea, because it's easier to control the temperature instead of measuring and compensating the temperature effect on a quartz.

It's not easier to use than a TCXO, choose whatever you like :slight_smile:

#15

DrDiettrich:
An OCXO (oven controlled...) works in a wide range of ambient temperatures, in detail in cold environment. I like that idea, because it's easier to control the temperature instead of measuring and compensating the temperature effect on a quartz.

Thanks. That's interesting. I can see how that would be more accurate.

#16