Multiplying millis() (a long) by a float?

Apologies if this has been asked before, I've searched but I cannot find anything.

So, the internal clock on my Arduino clone is absolute rubbish (loses around 6 ms per sec), but my DS3231 doesn't give me the milliseconds I need. I think I've found a solution whereby every five minutes I sync up the internal clock to the DS3231, but to do that I also need to stretch millis() to be less rubbish. I've run a regression which allows me to incorporate the temperature from the DS3231 into my calculation to improve this stretch, and it should perform pretty well (loses around 3ms per five minute block, which is absolutely fine).

BUT -- to do this, I need to multiply millis() -- a long -- by my floating point stretch factor (around 1.006 but varying as the temperature changes). I'm a bit stumped as to how to do this? I'm happy for it to wrap around to zero after a couple of months in the same way that millis() does.

The best I can come up with is some hack-job where I break millis() down into two ints, multiply each of those by (stretch factor-1 = .006) so that it's a manageable number, piece those back into a long, then add the original number. But even then, it's not clear to me how I would handle the multiplication on the more significant of the two ints, as it would presumably again need more digits than my float can handle.

is there anything neater? Thank you!

No need for floating point. Can all be done with integer maths.

Multiply your correction factor by 4,294,967,295 (or any power of two you require). Take the integer part.
Now do a 64bit integer multiply of millis() by this number and shift the result right by 32 bits (or whatever power of 2 you thought of).

More bits greater accuracy, but less maximum resolution. 32 is the most you can get away with while still maintaining a 32-bit result.

Edit: you don’t say how your correction factor is calculated. Ideally that wouldn’t involve any floating point either. Just fixed point integer as described above.

It would be easier just to add the missing milliseconds. You should also mind that the oscillator is influenced by the ambient temperature, it oscillates slower when the temperature is lower and ditto higher.

unsigned long compensatedMillis() {
  unsigned long res = millis();
  return res + ((res / 1000) * 6); //Add 6 millis for each second since boot
}

The code will work until millis() rollover.

Thank you both, these are both very helpful thoughts! I should add that 1.006 is an approximate number -- it has more decimals than this and is already calculated to incorporate the ambient temperature, so varies from one five-minute sync to the next, so I'm not sure if Danois90's approach would then work (also, I think it would lose some precision?).

@pcbbc -- that sounds like a great solution! I did wonder about that, but don't long longs eat up huge amounts of memory (I believe I have read that their implementation means that program sizes become very bloated, though I don't quite understand why).

philassheton:
Apologies if this has been asked before, I've searched but I cannot find anything.

So, the internal clock on my Arduino clone is absolute rubbish (loses around 6 ms per sec),

Same as the official Uno I'm afraid - years ago it changed from using a nice accurate quartz
crystal for the microcontroller clock and switched to a crappy ceramic resonator, for mysterious
and (IMO) misguided reasons. Probably a cost-cutting exercise, but the result is the board's not
accurate enough for many timing applications such as a simple stop-watch or interval timer.

One Uno I had still had the pads for a crystal as well as resonator, so I desoldered the resonator
and stuck a 16MHz crystal+caps in its place - I don't think the newer Uno's have the pads for this :frowning:

You could say that you need to add 1ms for each 1000/6=166,67ms. That would cause the approximate math to look like:

unsigned long compensatedMillis() {
  unsigned long ms = millis();
#ifdef ROLLOVER_SAFE
  static unsigned long compensate = 0, lastMillis = 0;
  unsigned long elapsedMillis = ms - lastMillis;
  if (elapsedMillis >= 1000) {
    unsigned long secs = elapsedMillis / 1000;
    compensate += (secs * 6);
    secs *= 1000;
    lastMillis += secs;
    elapsedMillis -= secs;
  }
  return ms + compensate + (elapsedMillis / 167);
#else
  return ms + ((ms / 1000) * 6) + ((ms % 1000) / 167);
#endif
}

This should have higher accuracy and even be rollover safe (for 2^32/6 rollovers).

EDIT: Corrected the code in case someone should find it.

philassheton:
So, the internal clock on my Arduino clone is absolute rubbish (loses around 6 ms per sec), but my DS3231 doesn't give me the milliseconds I need. I think I've found a solution whereby every five minutes I sync up the internal clock to the DS3231,

Sounds like it would be a lot simpler just to use the DS3231 and not bother with the Arduino millis() function.

If necessary (or useful) perhaps you could write your own myMillis() function that gives a value derived from the DS3231 and use that in place of the standard millis() function.

...R

Robin2:
Sounds like it would be a lot simpler just to use the DS3231 and not bother with the Arduino millis() function.

Isn't the problem that the DS3231 doesn't have an accessible ms? Just Y/M/D H:M:S?

Presumably the module has a 32KHz output though? Why not count pulses on that to provide the ms figure?
The actual pulse count is 32,768 per second, so...
ms = count * 1000 / 32768
or
ms = (count * 2000) >> 16

Thank you all for the input!!

@MarkT it’s a Mega2560 clone, but definitely has a terrible clock.

I’m at work so reading these little hurriedly. @Danois90 it feels like I lose accuracy when I divide elapsedMillis by 1000 before I then multiply back up? And i would have to divide by a larger number as I need something more like 1.006645

@pcbbc – Yes, that is it exactly. Perhaps I will take the plunge and try to implement this! My background is in mathematical modelling, not in hardware-level stuff, so my default was to try to find a clever numerical way of getting around this, but I think you’re right I’ll have to look into doing it that way. I think I have to write an interrupt routine on one of the digital inputs, is that right? Hopefully one of my few remaining digital ins is interruptable, as I don’t have many left after all my other connections (I’m wiring up a piano to detect keypresses).

philassheton:
@Danois90 it feels like I lose accuracy when I divide elapsedMillis by 1000 before I then multiply back up? And i would have to divide by a larger number as I need something more like 1.006645

I think my approach for compensation is as accurate as it gets in this context. If you want higher accuracy you will need external hardware.

I faced this problem. I chose the path of using a board that has a real crystal. But if you want to run with what you have, probably the best way would be to attach the 32kHz output of the DS3231 to an interrupt enabled input pin. Obviously, this will improve your synch accuracy by a factor of 2^15:1 compared with synch to PPS.

The problem with synchronizing the resonator is that it is not only inaccurate, but varies wildly with temperature. So the validity of any calibration that you do, with the expectation of synchronizing once per second, will be short lived. The granularity of a single second measurement is high, 1/1000. However it can be used for the next second because the temperature can't change that fast. But if you want microsecond accuracy, the measurement interval has to be longer to improve the granularity. I have a synch to GPS PPS calibration routine to achieve ~0.5 PPM, in order to do that it has to run for several minutes.

If you want to get creative, it is possible to derive timing from the 16U2 on the UNO because it uses a real crystal (to guarantee accurate baud rates, I think).

Some processors with an internal RTC allow access to a subseconds register, such as the STM32F103C8. It normally counts at 32kHz. The STM32 boards I have also use a real crystal. So you could use the subseconds register to synch milliseconds in software. This would just put the DS3231/32kHz/interrupt pin solution into one package, and using no I/O pins. However, that RTC is not automatically temperature compensated like the DS3231.

philassheton:
I think I have to write an interrupt routine on one of the digital inputs, is that right? Hopefully one of my few remaining digital ins is interruptable, as I don't have many left after all my other connections (I'm wiring up a piano to detect keypresses).

That's one way to do it.

The other way is to use one of the 16-bit timers in the Mega...
Timer 1 – Port D, Bit 6
Timer 3 – Port E, Bit 6
Timer 4 – Port H, Bit 7
Timer 5 – Port L, Bit 2

I'm not that familiar with the 2560, but at least one (if not all) should be able to be configured to count pulses on one of those pins entirely autonomously.

@aarg Thank you! It sounds like one extra connection to the DS3231 will be the best solution given that I have that all working now and hooked up.

@pcbbc – This is a foreign language to me, but if it can count the pulses automatically for me, it sounds exactly what I need!! And far less bulky codewise. I will have a read after work to try to understand what these ports are and how to access them.

Two follow-up thoughts on top of last message:

@aarg regarding temperature calibration, I ran a test last night, recording temperatures from DS3231 and internal clock speeds from the Arduino, calibrating every five minutes, and analysing the data from that, it looks like that does allow me to keep track to within about 8ms if I recalibrate every five minutes (including accounting for temperature). Anything up to about 15ms (or 30ms at a push) drift is no problem for me.

@pcbbc if it's a sixteen bit timer, does that mean that it will overflow roughly every two seconds at 32kHz? It occurs to me, that that might be problematic...

philassheton:
@aarg Thank you! It sounds like one extra connection to the DS3231 will be the best solution given that I have that all working now and hooked up.

@pcbbc – This is a foreign language to me, but if it can count the pulses automatically for me, it sounds exactly what I need!! And far less bulky codewise. I will have a read after work to try to understand what these ports are and how to access them.

Yeah, pcbbc reminded me of that. If you want to delve into the times between 32kHz pulses, you have to do interpolation. I believed that is what you are trying to do. In that case, you need two software components: an interpolater, which uses the CPU measured time from the last received 32k pulse, and a calibrator, which runs over a longer time scale, and produces a running calibration offset for the interpolator to use.

Alternatively, you could just do the first part and use raw CPU timing from the last 32k pulse, if the error is in fact not too large.

32kHz pulses come in every ~30us, so the needed granularity for the latter direct interpolation is very low. In other words, you can synchronize every 30us (at every interrupt) and even with a very wild CPU clock, be accurate enough by simply counting microseconds elapsed since the last pulse. But, to extend that principle to an interface that offers you a millis-like timing value, is another thing. My math is sometimes fuzzy in the morning… the straightforward way would be to add the pulse time to an accumulator each time, but it would have to be scaled because the exact inverse of 32768 is an 35 digit binary number. Then you would just add the microseconds past the pulse and the accumulator value, to obtain the current time in microseconds. Milliseconds might be easier, since in fact, the granularity of the 32kHz is about 33 times smaller than milliseconds.

You could also just increment a counter with every interrupt, do some math knowing that it represents the number of 30.517578125us pulses have elapsed from some start time. Maybe I’ve missed some really obvious simple solution.

philassheton:
@pcbbc -- that sounds like a great solution! I did wonder about that, but don't long longs eat up huge amounts of memory (I believe I have read that their implementation means that program sizes become very bloated, though I don't quite understand why).

That is correct. Usually you arrange for an interrupt when the timer overflows. This way you only have an interrupt every 1 or 2 seconds, instead of every approx 30us.

The hardware counts the pulses on the digital pin, and you can read them at any time via the timer count register: TCNTx

So you'd use software to keep track of the Y/M/D H:M:S (which you can initialize from the RTC at startup) via the interrupt
And you'd read TCNTx 16-bit register to get the elapsed 32.768kHz ticks into the current second when you want the time

Do I assume you are trying to measure an interval to ms() accuracy?
In that case you do not need Y/M/D H:M:S. Just use the interrupt to keep track of elapsed total seconds and the TCNTx count register to calculate the milliseconds part.

Also, from a hardware standpoint, The DS3231 needs no I2C interface or backup battery in order to produce 32kHz pulses, as they are enabled by default. Just power and ground.

Here's a very basic example of how to count pulses on pin 5 of an Uno using Timer/Counter 1...

void setup()
{
  Serial.begin(9600);

  // Configure T1 to count pulses on pin 5
  pinMode(5, INPUT);
  TCCR1A = 0x00;
  TCCR1B = 0x07;
  TCNT1 = 0;
}

void loop()
{
  // Output current count of pulses
  Serial.println(TCNT1, HEX);
}

Apologies my replies are sparse and short as still at work.

@pcbbc. This is perfect! And solves all my worries. One reason I had avoided counting the pulses was because it felt like too many interrupts, but as you say, if the timer can then cause an interrupt that is spot on.

I want to record all values of date and time down to millisecond-ish accuracy (I'm actually only recording it to 4ms accuracy (8 bits per second), but a bit less than that could still work). To give the context -- I write music on piano and often wish I could go back and work out what notes I just played, so I've covered my piano keys with copper tape and hooked them up to an arduino and want it to record every note I play. I need the times to be accurate as I have also written some R code to translate the recorded notes into a video animation of the keys being pressed that is synced to my little audio recordings that I make with my phone. I have it all working, but at present it is having to watch the DS3231 every second for when the second flips and resync the clock each second to that, which is not great.

@aarg this is super cool! But I think that would only be if I needed microsecond accuracy?