realtime clock, microseconds, etc.

Hi DCB, just in case you were not aware of it, there is a DateTime library in the playground that keeps track of time in seconds that doesn't require modifying wiring.c

It's a software real time clock that can be set to the actual time, or if set to zero at startup will return the number of seconds since the sketch started.

The playground article is here: Arduino Playground - DateTime

Sure, I remember that, was glad to see it, but without interrupt support for the time model, you had to be sure to call it every so often or else it could miss a millis() turnover, IIRC. With this, the timer0 interrupt keeps track of actual run time internally for a ridiculously long amount of time.

Ok, maybe it makes more sense if I fill in the pieces a little more. With the previous changes to wiring.c, it becomes possible to track both seconds and microseconds.

Here is a sample sketch, though these functions might belong in a library (or also in wiring) eventually:

extern volatile unsigned long timer0_count;
extern volatile unsigned long timer0_seconds;

void setup(){
  Serial.begin(9600);
}

unsigned long mil, t0sec, t0us ;
void loop(){
 cli();
 t0us=microSeconds();
 mil=millis();
 t0sec=seconds();
 sei();
 Serial.print(t0sec);
 Serial.print(" ");
 Serial.print(mil);
 Serial.print(" ");
 Serial.print(t0us);
 Serial.println(" ");
 delay(500);
}

//keep track of seconds over a period of 136 years
unsigned long seconds(){
  unsigned long m, s;
  uint8_t oldSREG = SREG;
  cli();
  m = timer0_count;
  s = timer0_seconds;
  SREG = oldSREG;
  m=(m*( 8 * clockCyclesPerMicrosecond())/125000)+(s);                  
  return m;
}

//microseconds timestamp recycles every 128 seconds
unsigned long microSeconds(){
  unsigned long m, t;
  uint8_t oldSREG = SREG;
  cli();
  t = TCNT0;
  m = timer0_count;
  SREG = oldSREG;
  return ((m << 8) + t) * 4;
}

and here is some output:
0 0 88
0 507 507960
1 1021 1022020
1 1537 1537108
2 2052 2052184
2 2567 2567228
3 3082 3082308
3 3597 3597384
...
126 126051 126051396
126 126572 126572632
127 127094 127094840
127 127616 127616088
128 128137 137272
128 128655 655428
129 129173 1173572
129 129692 1692748
130 130212 2212924
130 130732 2732124

So you can see that at 128 seconds, the microseconds timestamp rolls over and represents the last three digits of the milliseconds column. You can use microSeconds() to time events up to 128 seconds long (do the rollover trick to see if you need to subtract from 128 seconds), and you can do elapsed seconds based calculations like nothing. You'll be long gone before that rolls over :slight_smile:

And one more bit of core functionality to consider. A user defined callback, which would allow for building an event scheduler (will post that later):

here is a possible wiring.c implementation:

...
void (*userDefinedTimer0Handler)(void) = NULL;

SIGNAL(SIG_OVERFLOW0)
{
    timer0_count++;
    while(timer0_count>=125000){
      timer0_count-=125000;
      timer0_seconds+= 8* clockCyclesPerMicrosecond();
    }
    if(userDefinedTimer0Handler)
      userDefinedTimer0Handler();
}
...

and a rough cut script that makes use of it

extern void  (*userDefinedTimer0Handler)(void);

volatile unsigned long  t0ovfX3;
void myTimer0Callback(){
  t0ovfX3 += 3;
}

void setup(){
  userDefinedTimer0Handler=myTimer0Callback;
  Serial.begin(9600);
}

void loop(){
 Serial.println(t0ovfX3);
 delay(500);
}

without interrupt support for the time model, you had to be sure to call it every so often or else it could miss a millis() turnover

The DateTime code will not miss a turnover unless the sketch waits over four years between each call to get the time. Can you think of any application where a sketch would wait longer than this between intervals to check the time?

And one more bit of core functionality to consider. A user defined callback, which would allow for building an event scheduler

There is code to do date and time scheduling using callbacks posted here:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1217881285

this code uses the DateTime library so does not need changes to wiring.c. As above, no events get lost as long as the interval between successive scheduled events occurs at least every four years

FWIW there was code posted here http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1210889307/8#8 that updated DateTime by changing the interrupt handler in wiring.c (that code was for 0011) but I didn't think the four year interval between updates would be a problem, do you?

no, 4 years isn't a problem. I do have use for a milliseconds resolution callback function (interrupt debouncing and ???) though, as well as getting a handle on microseconds.

There is nothing wrong with leveraging wiring.c either as far as I know. The core files are not in their final form, and subject to change based on community input. And this isn't much more complicated than the current functionality.

I'm sure if you had a seconds function available that was good for 136 years you would have used that as a base for the datetime library, no?

There's a number of folks wanting microsecond timestamps as well, would like to see that supported in the core. If timer0 can handle both micro timing and long term timing needs then why not?

It appears on the surface that the main difference in scheduling approaches is that with this version it is interrupt driven. At about 1ms resolution you can schedule the re-enabling of an input pin, or the playing of the next note in a background pwm music generator, without having to poll millis(). And being interrupt driven (need wiring.c change for that on timer0) it could also serve as the basis for a larger scale scheduler (i.e. time to wake up, or go to the dentist).

It looks like you must always call delay/wait in dtalarms to keep the scheduler active. Which is a workable solution in many cases, but would be more ideally handled in the background in many cases as well.

The real caveat with the interrupt approach is keeping the callback function small, i.e. just re-enable the pin in question or perform the pwm frequency change and re-schedule the next change. But wiring.c is fair game to that end, IMHO.

I certainly agree that your approach is valid, and I would not want to discourage your developments. But I do have a different perspective on the relative merits of the alternatives so perhaps it could be useful for me to clarify.

I had considered modifying wiring.c (mellis had offered to make the change mentioned in the link above) that would have added a seconds counter to the interrupt handler. I decided not to do this because I though there was no practical advantage in doing so. The code as is will not roll over for 138 years from the time the seconds counter is zeroed. And calling a timekeeping function at least once every four years does not seem like an impediment to any Arduino user.
My background in software engineering has taught me that every line of code added can bring significant baggage in future maintenance and I think solutions with the lighter touch are more appropriate to the Arduino environment. YMMV

Anyway, I will watch your work on real time scheduling with interest.

Good luck!

I'm still digesting all this, but I thought I'd point out that the code for the microSeconds seems to assume (and rely on) a 16MHz clock. If I understand this correctly (and I may not), with a prescale value of 64, the value of TCNT0 increments every (1/16) * 64 = 4us, hence the "* 4" in the expression. However, this will not be valid at other frequencies, will it? Perhaps the "4" should be replaced with "(prescale/clockCyclesPerMicrosecond())"?

Maybe, it looks like the same prescaler for 8 or 16mhz in the existing code. Can't currently test 8mhz though (don't have an 8mhz *duino handy) otherwise I would have tested it. Sicne we are dealing with microseconds though, it should probably be another #define based on F_CPU if it needs fixing.

http://svn.berlios.de/viewcvs/arduino/trunk/hardware/cores/arduino/wiring.c?view=markup

I'd love to have a micros() function that returns the number of microseconds since the sketch started. It's also important that the core timing functions work reliably at 16 and 8 MHz, and accurately (though not necessarily precisely) at other speeds. It could also be nice to have built-in second(), minute(), hour(), day(), month() etc. functions that return the current date and time. (I imagine that they would assume the sketch started at, say, midnight on Jan 1, 2000, but that there would be a setTime() that would take the number of milliseconds since the epoch, for example.) I'm not so worried about how many years it is between overflows of the millis() counter.

Any thoughts on the best way to achieve these goals?

Returning the elapsed microseconds can be done in a number of ways, here are three approaches that come to mind:

The first two optimize for applications that will call this function many thousands of times a second. Using hardware timer1 (or timer3) would be a good way of supporting this but would cause the loss of other important functionality so I would not like to see this in the core. But perhaps it could be a useful library add-on for specialized applications that need precision micro timing.

Accumulating the count in an interrupt could be a way to go but this adds processing overhead to every sketch whether the functionality is needed or not.

I favor an approach that adds little or no overhead for the vast majority of users that don't need an elapsed microsecond count. This could be done by calculating the count only when the application makes a call to get the value. The function would be something like:

#define TIMER0_TO_MICROSECONDS (insert code here to convert each timer0 count to microseconds); 

unsigned long micros()
{
   return (millis() * 1000) + TIMER0_TO_MICROSECONDS;
}

As to date and time functions, I would be happy to adapt the playground DateTime library for use in the core. Certainly the change to start at Jan 1 2000 (or any specific date) would be simple to do.

FYI, my proposal adds zero interrupt processing overhead to what is currently implemented until you bring in the scheduler library. And you don't have to look far for people wanting better than millisecond timing. It doesn't exist so maybe that is why it isn't more in demand.

I don't understand what functionality you are talking about losing mem? I'm trying to add functionality.

But I do think additional microseconds and days/months/years (which I have never had a need for) functionality should be only brought in on demand. So with timer0 keeping track of seconds + cycles and some pre-arranged hooks for the library, the additional code can be done on demand.

So if I had my druthers, the top of wiring.c would look like below, and a library would get brought in for micros, elapsedMicros, seconds. And the datetime library could be based on the seconds function to eliminate any possibility of a missed rollover and add its own userDefinedTimer0Handler for scheduling small tasks. Though this level of scheduling may be better suited for debouncing, background music, ??? and not (hey time to send all the log data now). It may be that mems use case is a better fit for a scheduler that requires polling, where I want to fire and forget the next note to play and the re-enabling of a button, etc.

...

#include "wiring_private.h"

volatile unsigned long timer0_count = 0;
volatile unsigned long timer0_seconds = 0;
void (*userDefinedTimer0Handler)(void) = NULL;

SIGNAL(SIG_OVERFLOW0)
{
    timer0_count++;
    while(timer0_count>=125000){
      timer0_count-=125000;
      timer0_seconds+= 8* clockCyclesPerMicrosecond();
    }
    if(userDefinedTimer0Handler)
      userDefinedTimer0Handler();

}

unsigned long millis(){
      unsigned long m, s;
      uint8_t oldSREG = SREG;
      cli();
      m = timer0_count;
      s = timer0_seconds;
      SREG = oldSREG;
      m=(m*( 8* clockCyclesPerMicrosecond())/125)+(s*1000);                          
        return m;
}
...

Anyone want to take a shot at code for calculating microseconds from timer0 overflow counts and TCNT0? Again, it should work at any CPU speed, although it only need to be precise for 8 MHz and 16 MHz.

It looks like Don Kinzer is off to a good start with it (in hptics):
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1226257074/1#1

though with the divisions it is a bit more computing than I'm proposing, which at it's core could be expressed as:
((m << 8) + t) <<2;

I wanted to keep it light so you can get a new microseconds timestamp from an interrupt without much concern, so it overflows every 128 seconds internally instead of computing the seconds portion as well, but just adding timer0_seconds * 1000000 will get you there.

But I figure being able to track any number of signals at low microsecond accuracy with a period of up to just over 2 minutes is pretty useful.

Like so much of microprocessors, always a compromise :slight_smile:

I agree with dcb that there seems to be a great deal of interest in built-in microsecond and "macrosecond" timing. Thanks for the thought you've put into this!

I'm a little uncomfortable with the having values that overflow on non-word boundaries (like the 128 seconds proposed above). It makes it difficult to compute time deltas. Remember the trouble we had because 0011's millis() didn't overflow on the word boundary? I think it's important to be able to write delta expressions like:

while (micros() - start > 20) // wait 20 microseconds

without worrying about how the overflow will affect the calculation.

David, I like the way the current wiring.c implementation handles SIG_OVERFLOW0. I don't see that we need to change it to build a micros() function that overflows at the 32-bit boundary. This should work at both 16 and 8 MHz without changing wiring.c at all:

unsigned long micros()
{
unsigned long u;
uint8_t oldSREG = SREG;
cli();
u = 1000 * timer0_millis + (timer0_clock_cycles + 64UL * TCNT0) / clockCyclesPerMicrosecond();
SREG = oldSREG;
return u;
}

(Don't be alarmed by all the multiplies and divides. All but one will/can be optimized as bit shifts. I agree with dcb's strategy of keeping things as streamlined as possible.)

Mikal

I would also like to throw my support behind adding a "millis_overflow" counter to the SIG_OVERFLOW handler. This would be minimally invasive:

volatile unsigned long timer0_clock_cycles = 0;
volatile unsigned long timer0_millis = 0;
volatile unsigned long timer0_millis_overflow = 0;

SIGNAL(SIG_OVERFLOW0)
{
// timer 0 prescale factor is 64 and the timer overflows at 256
timer0_clock_cycles += 64UL * 256UL;
while (timer0_clock_cycles > clockCyclesPerMicrosecond() * 1000UL) {
timer0_clock_cycles -= clockCyclesPerMicrosecond() * 1000UL;
timer0_millis++;
** if (timer0_millis == 0)**
** timer0_millis_overflow++;**
}
}

but would solve many problems with "macrosecond" timekeeping. With this technology, anyone can track total micro/milliseconds elapsed without having to periodically call some kind of library refresh function every 49 days or 4 years or whatever. Overflow measured in centuries.

Mikal

David, to my thinking, date/time functions like hour(), day(), etc. don't belong in the wiring.c "kernel". Tracking "clock-based" time seems to be a different problem than tracking "elapsed" time. While nearly every Arduino project depends on the latter -- millis() and delay(), etc. -- the need for "clock" time seems more limited. In my opinion, this should be placed in a library -- like mem's DateTime! :slight_smile:

Mikal

The code dcb suggested for a microseconds function has the same problem that I also had in my first proposed code for hpticks(). The problem is that the if the timer overflows between the time the cli instruction is executed and when the TCNT0 register is read, the result will be off by 256 timer ticks.

This problem can be resolved by checking for the timer overflow flag being set and the value of TCNT0 being 0, indicating that the timer just rolled over. The difficulty with implementing is is that the name of the register containing TOV0 is different depending on which AVR is being used. A suggested solution, which addresses the register name difference, is shown below.

// define the timer interrupt flag register if necessary
#if !defined(TIFR0)
  #if defined(TIFR)
    #define TIFR0 TIFR
  #else
    #error AVR device not supported
  #endif
#endif

unsigned long microSeconds()
{
  unsigned long m, t;
  uint8_t oldSREG = SREG;
  cli();
  t = TCNT0;
  if ((TIFR0 & _BV(TOV0)) && (t == 0))
    t = 256;
  m = timer0_count;
  SREG = oldSREG;
  return ((m << 8) + t) * 4;
}

The CPU speed dependency can be removed by replacing the return value computation with that shown below. This has the limitation, however, of working correctly only when F_CPU is an integral multiple of 10^6.

  return ((m << 8) + t) * (64 / (F_CPU / 1000000L));

Thanks for the interesting analysis, Don.

The CPU speed dependency can be removed by replacing the return value computation with that shown below. This has the limitation, however, of working correctly only when F_CPU is an integral multiple of 10^6.

 return ((m << 8) + t) * (64 / (F_CPU / 1000000L));

Doesn't it also rely on F_CPU being divisible into 64 million? I bring this up only because there has been some discussion of supporting processors at 20MHz and it seems like this might cause a problem if F_CPU were 20000000.

Mikal