realtime clock, microseconds, etc.

no, 4 years isn't a problem. I do have use for a milliseconds resolution callback function (interrupt debouncing and ???) though, as well as getting a handle on microseconds.

There is nothing wrong with leveraging wiring.c either as far as I know. The core files are not in their final form, and subject to change based on community input. And this isn't much more complicated than the current functionality.

I'm sure if you had a seconds function available that was good for 136 years you would have used that as a base for the datetime library, no?

There's a number of folks wanting microsecond timestamps as well, would like to see that supported in the core. If timer0 can handle both micro timing and long term timing needs then why not?

It appears on the surface that the main difference in scheduling approaches is that with this version it is interrupt driven. At about 1ms resolution you can schedule the re-enabling of an input pin, or the playing of the next note in a background pwm music generator, without having to poll millis(). And being interrupt driven (need wiring.c change for that on timer0) it could also serve as the basis for a larger scale scheduler (i.e. time to wake up, or go to the dentist).

It looks like you must always call delay/wait in dtalarms to keep the scheduler active. Which is a workable solution in many cases, but would be more ideally handled in the background in many cases as well.

The real caveat with the interrupt approach is keeping the callback function small, i.e. just re-enable the pin in question or perform the pwm frequency change and re-schedule the next change. But wiring.c is fair game to that end, IMHO.

I certainly agree that your approach is valid, and I would not want to discourage your developments. But I do have a different perspective on the relative merits of the alternatives so perhaps it could be useful for me to clarify.

I had considered modifying wiring.c (mellis had offered to make the change mentioned in the link above) that would have added a seconds counter to the interrupt handler. I decided not to do this because I though there was no practical advantage in doing so. The code as is will not roll over for 138 years from the time the seconds counter is zeroed. And calling a timekeeping function at least once every four years does not seem like an impediment to any Arduino user.
My background in software engineering has taught me that every line of code added can bring significant baggage in future maintenance and I think solutions with the lighter touch are more appropriate to the Arduino environment. YMMV

Anyway, I will watch your work on real time scheduling with interest.

Good luck!

I'm still digesting all this, but I thought I'd point out that the code for the microSeconds seems to assume (and rely on) a 16MHz clock. If I understand this correctly (and I may not), with a prescale value of 64, the value of TCNT0 increments every (1/16) * 64 = 4us, hence the "* 4" in the expression. However, this will not be valid at other frequencies, will it? Perhaps the "4" should be replaced with "(prescale/clockCyclesPerMicrosecond())"?

Maybe, it looks like the same prescaler for 8 or 16mhz in the existing code. Can't currently test 8mhz though (don't have an 8mhz *duino handy) otherwise I would have tested it. Sicne we are dealing with microseconds though, it should probably be another #define based on F_CPU if it needs fixing.

http://svn.berlios.de/viewcvs/arduino/trunk/hardware/cores/arduino/wiring.c?view=markup

I'd love to have a micros() function that returns the number of microseconds since the sketch started. It's also important that the core timing functions work reliably at 16 and 8 MHz, and accurately (though not necessarily precisely) at other speeds. It could also be nice to have built-in second(), minute(), hour(), day(), month() etc. functions that return the current date and time. (I imagine that they would assume the sketch started at, say, midnight on Jan 1, 2000, but that there would be a setTime() that would take the number of milliseconds since the epoch, for example.) I'm not so worried about how many years it is between overflows of the millis() counter.

Any thoughts on the best way to achieve these goals?

Returning the elapsed microseconds can be done in a number of ways, here are three approaches that come to mind:

The first two optimize for applications that will call this function many thousands of times a second. Using hardware timer1 (or timer3) would be a good way of supporting this but would cause the loss of other important functionality so I would not like to see this in the core. But perhaps it could be a useful library add-on for specialized applications that need precision micro timing.

Accumulating the count in an interrupt could be a way to go but this adds processing overhead to every sketch whether the functionality is needed or not.

I favor an approach that adds little or no overhead for the vast majority of users that don't need an elapsed microsecond count. This could be done by calculating the count only when the application makes a call to get the value. The function would be something like:

#define TIMER0_TO_MICROSECONDS (insert code here to convert each timer0 count to microseconds); 

unsigned long micros()
{
   return (millis() * 1000) + TIMER0_TO_MICROSECONDS;
}

As to date and time functions, I would be happy to adapt the playground DateTime library for use in the core. Certainly the change to start at Jan 1 2000 (or any specific date) would be simple to do.

FYI, my proposal adds zero interrupt processing overhead to what is currently implemented until you bring in the scheduler library. And you don't have to look far for people wanting better than millisecond timing. It doesn't exist so maybe that is why it isn't more in demand.

I don't understand what functionality you are talking about losing mem? I'm trying to add functionality.

But I do think additional microseconds and days/months/years (which I have never had a need for) functionality should be only brought in on demand. So with timer0 keeping track of seconds + cycles and some pre-arranged hooks for the library, the additional code can be done on demand.

So if I had my druthers, the top of wiring.c would look like below, and a library would get brought in for micros, elapsedMicros, seconds. And the datetime library could be based on the seconds function to eliminate any possibility of a missed rollover and add its own userDefinedTimer0Handler for scheduling small tasks. Though this level of scheduling may be better suited for debouncing, background music, ??? and not (hey time to send all the log data now). It may be that mems use case is a better fit for a scheduler that requires polling, where I want to fire and forget the next note to play and the re-enabling of a button, etc.

...

#include "wiring_private.h"

volatile unsigned long timer0_count = 0;
volatile unsigned long timer0_seconds = 0;
void (*userDefinedTimer0Handler)(void) = NULL;

SIGNAL(SIG_OVERFLOW0)
{
    timer0_count++;
    while(timer0_count>=125000){
      timer0_count-=125000;
      timer0_seconds+= 8* clockCyclesPerMicrosecond();
    }
    if(userDefinedTimer0Handler)
      userDefinedTimer0Handler();

}

unsigned long millis(){
      unsigned long m, s;
      uint8_t oldSREG = SREG;
      cli();
      m = timer0_count;
      s = timer0_seconds;
      SREG = oldSREG;
      m=(m*( 8* clockCyclesPerMicrosecond())/125)+(s*1000);                          
        return m;
}
...

Anyone want to take a shot at code for calculating microseconds from timer0 overflow counts and TCNT0? Again, it should work at any CPU speed, although it only need to be precise for 8 MHz and 16 MHz.

It looks like Don Kinzer is off to a good start with it (in hptics):
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1226257074/1#1

though with the divisions it is a bit more computing than I'm proposing, which at it's core could be expressed as:
((m << 8) + t) <<2;

I wanted to keep it light so you can get a new microseconds timestamp from an interrupt without much concern, so it overflows every 128 seconds internally instead of computing the seconds portion as well, but just adding timer0_seconds * 1000000 will get you there.

But I figure being able to track any number of signals at low microsecond accuracy with a period of up to just over 2 minutes is pretty useful.

Like so much of microprocessors, always a compromise :slight_smile:

I agree with dcb that there seems to be a great deal of interest in built-in microsecond and "macrosecond" timing. Thanks for the thought you've put into this!

I'm a little uncomfortable with the having values that overflow on non-word boundaries (like the 128 seconds proposed above). It makes it difficult to compute time deltas. Remember the trouble we had because 0011's millis() didn't overflow on the word boundary? I think it's important to be able to write delta expressions like:

while (micros() - start > 20) // wait 20 microseconds

without worrying about how the overflow will affect the calculation.

David, I like the way the current wiring.c implementation handles SIG_OVERFLOW0. I don't see that we need to change it to build a micros() function that overflows at the 32-bit boundary. This should work at both 16 and 8 MHz without changing wiring.c at all:

unsigned long micros()
{
unsigned long u;
uint8_t oldSREG = SREG;
cli();
u = 1000 * timer0_millis + (timer0_clock_cycles + 64UL * TCNT0) / clockCyclesPerMicrosecond();
SREG = oldSREG;
return u;
}

(Don't be alarmed by all the multiplies and divides. All but one will/can be optimized as bit shifts. I agree with dcb's strategy of keeping things as streamlined as possible.)

Mikal

I would also like to throw my support behind adding a "millis_overflow" counter to the SIG_OVERFLOW handler. This would be minimally invasive:

volatile unsigned long timer0_clock_cycles = 0;
volatile unsigned long timer0_millis = 0;
volatile unsigned long timer0_millis_overflow = 0;

SIGNAL(SIG_OVERFLOW0)
{
// timer 0 prescale factor is 64 and the timer overflows at 256
timer0_clock_cycles += 64UL * 256UL;
while (timer0_clock_cycles > clockCyclesPerMicrosecond() * 1000UL) {
timer0_clock_cycles -= clockCyclesPerMicrosecond() * 1000UL;
timer0_millis++;
** if (timer0_millis == 0)**
** timer0_millis_overflow++;**
}
}

but would solve many problems with "macrosecond" timekeeping. With this technology, anyone can track total micro/milliseconds elapsed without having to periodically call some kind of library refresh function every 49 days or 4 years or whatever. Overflow measured in centuries.

Mikal

David, to my thinking, date/time functions like hour(), day(), etc. don't belong in the wiring.c "kernel". Tracking "clock-based" time seems to be a different problem than tracking "elapsed" time. While nearly every Arduino project depends on the latter -- millis() and delay(), etc. -- the need for "clock" time seems more limited. In my opinion, this should be placed in a library -- like mem's DateTime! :slight_smile:

Mikal

The code dcb suggested for a microseconds function has the same problem that I also had in my first proposed code for hpticks(). The problem is that the if the timer overflows between the time the cli instruction is executed and when the TCNT0 register is read, the result will be off by 256 timer ticks.

This problem can be resolved by checking for the timer overflow flag being set and the value of TCNT0 being 0, indicating that the timer just rolled over. The difficulty with implementing is is that the name of the register containing TOV0 is different depending on which AVR is being used. A suggested solution, which addresses the register name difference, is shown below.

// define the timer interrupt flag register if necessary
#if !defined(TIFR0)
  #if defined(TIFR)
    #define TIFR0 TIFR
  #else
    #error AVR device not supported
  #endif
#endif

unsigned long microSeconds()
{
  unsigned long m, t;
  uint8_t oldSREG = SREG;
  cli();
  t = TCNT0;
  if ((TIFR0 & _BV(TOV0)) && (t == 0))
    t = 256;
  m = timer0_count;
  SREG = oldSREG;
  return ((m << 8) + t) * 4;
}

The CPU speed dependency can be removed by replacing the return value computation with that shown below. This has the limitation, however, of working correctly only when F_CPU is an integral multiple of 10^6.

  return ((m << 8) + t) * (64 / (F_CPU / 1000000L));

Thanks for the interesting analysis, Don.

The CPU speed dependency can be removed by replacing the return value computation with that shown below. This has the limitation, however, of working correctly only when F_CPU is an integral multiple of 10^6.

 return ((m << 8) + t) * (64 / (F_CPU / 1000000L));

Doesn't it also rely on F_CPU being divisible into 64 million? I bring this up only because there has been some discussion of supporting processors at 20MHz and it seems like this might cause a problem if F_CPU were 20000000.

Mikal

"The code dcb suggested for a microseconds function has the same problem that I also had "

Absolutely right, I have since been paying much closer attention to your TCNT0 handling Don :slight_smile:

Doesn't it also rely on F_CPU being divisible into 64 million? I bring this up only because there has been some discussion of supporting processors at 20MHz and it seems like this might cause a problem if F_CPU were 20000000.

Quite right. I overlooked that aspect of it. In my implementation of a higher precision timing function (see the link in my previous post), the function returns Timer0 ticks. This mitigates the 20MHz problem or, at least, defers it until conversion to microseconds is later done.

The Arduino-like device that I'm testing runs at 20MHz and my hpticks() function works correctly on it (the second attempt, at least). One issue to consider is that the suggested implementation of a microseconds function has a resolution of F_CPU / 64 since it is based on Timer0 ticks and Timer0 is clocked at 1/64th of the CPU frequency. Although a microseconds() function may be more aesthetically pleasing, the implementation essentially "wastes" a portion of the 32-bit value range. A function that returns Timer0 ticks will have the same resolution as a microseconds() function but will have a larger useful range. Moreover, the range will be constant irrespective of the CPU speed.

For measuring elapsed time, you can still think in terms of microseconds but convert the desired number of microseconds to Timer0 ticks (with either rounding or truncation as needed) before comparing it to the difference between two readings.

Hey guys, very nice work so far!

I would like to include a micros() function in the core if you think it's accurate / precise enough to be useful (particularly at 16 and 8 MHz). Can anyone run through an analysis of the resolution you could get from Don's latest function?

I think a ticks() function is too confusing to include in the core, although it would be a great thing to post on the playground: Arduino Playground - GeneralCodeLibrary.

The second(), minute(), hour(), etc. functions are not as high of a priority.

I don't think we need to worry about programs that run for more than a week or two. Beyond that, it's okay if you have to do some extra work to make things reliable (e.g. call a function every few days).

Does anyone want to summarize this discussion for the developers mailing list: http://mail.arduino.cc/mailman/listinfo/developers_arduino.cc?

Ok, I borrowed Dons hptics function (since it didn't require a change to wiring.c) and his latest TCNT0 handling and the tics2microseconds code (it is the micros function in the code block below)

The basic test harness is below. It uses timer2 to keep timer0 honest, and keeps track of the total differences between calls to micros()

16mhz atmega168
t2ms   t0ms   micros    sum(micros + prev micros)
...   
267500 267500 267500560 267500560 
268000 268000 268000272 268000272 
268500 268500 65552     65552 
269000 269000 565264    565264 
...

8mhz atmega168 (no crystal, lilypad bootloader&fuses)
t2ms   t0ms   micros    sum(micros + prev micros)   
...
536000 536000 536000560 536000560 
536500 536500 536500272 536500272 
537001 537001 131120    131120 
537501 537501 630832    630832 
...

So given the 1 to 1 relation of micros and the sum of changes to micros, and their agreement with millis and timer2, I'm fairly confident that this thing works pretty well :slight_smile: If there was noise from TCNT0 or elsewhere, I would expect it to show up when I subtract the current micros() from the last micros() and accumulate those differences, but it does not seem to be there.

Do note it rolls over at ~260 seconds at 16mhz. Which isn't so bad considering you can measure the distance to the moon in 3 seconds with a laser or time a sonic echo from 27 miles away with that range.

I do want to do some performance tests with the above changes and the wiring.c changes though, if you wind up polling microseconds then you need a fast microseconds. But we definitely have some good stuff to work with here.

extern volatile unsigned long timer0_clock_cycles;
extern volatile unsigned long timer0_millis;

unsigned long micros(){
  uint16_t t0;
  unsigned long clock_cycles;
  unsigned long millis;

  uint8_t sreg = SREG;
  cli();
  
  t0 = TCNT0;
  if ((TIFR0 & _BV(TOV0)) && (t0 == 0))
    t0 = 256;

  // get a snapshot of the other timer values
  clock_cycles = timer0_clock_cycles;
  millis = timer0_millis;
  SREG = sreg;

  // compute the number of Timer0 ticks represented by the data
  unsigned long t0_ticks = (clock_cycles / 64) + (millis * (1000L * clockCyclesPerMicrosecond() / 64)) + t0;
  return ((t0_ticks) * 64L / (F_CPU / 1000000L));
} 



volatile unsigned long timer2_clock_cycles = 0;
volatile unsigned long timer2_millis = 0;
ISR(TIMER2_OVF_vect){
// timer 2 prescale factor is 64 and the timer overflows at 256
  timer2_clock_cycles += 64UL * 256UL;
  while (timer2_clock_cycles > clockCyclesPerMicrosecond() * 1000UL) {
    timer2_clock_cycles -= clockCyclesPerMicrosecond() * 1000UL;
    timer2_millis++;
  }
}

unsigned long millis2(){
        unsigned long m;
      uint8_t oldSREG = SREG;
      cli();
      m = timer2_millis;
      SREG = oldSREG;
      return m;
}

void init2(){
      // on the ATmega168, timer 0 is also used for fast hardware pwm
      // (using phase-correct PWM would mean that timer 0 overflowed half as often
      // resulting in different millis() behavior on the ATmega8 and ATmega168)
#if defined(__AVR_ATmega168__)
        TCCR2A=1<<WGM20|1<<WGM21;
//      sbi(TCCR2A, WGM21);
//      sbi(TCCR2A, WGM20);
#endif  
      // set timer 0 prescale factor to 64
#if defined(__AVR_ATmega168__)
      TCCR2B=1<<CS22;
//      sbi(TCCR2B, CS21);
//      sbi(TCCR2B, CS20);
#else
      TCCR2=1<<CS21|1<<CS20
#endif
      // enable timer 0 overflow interrupt
#if defined(__AVR_ATmega168__)
      TIMSK2|=1<<TOIE2;
#else
      TIMSK|=1<<TOIE2;
#endif

}


void setup(){
  init2();
  Serial.begin(9600);
}

unsigned long nextMillis=500;
unsigned long microsSum;
unsigned long lastMicros;

void loop(){
  while(millis2()<nextMillis);
  unsigned long micros0;
  unsigned long millis0;
  unsigned long millisb;
  micros0=micros();
  millis0=millis();
  millisb=millis2();
  microsSum=microsSum + (micros0-lastMicros);
  lastMicros = micros0;
  Serial.print(millisb);
  Serial.print(" ");
  Serial.print(millis0);
  Serial.print(" ");
  Serial.print(micros0);
  Serial.print(" ");
  Serial.print(microsSum);
  Serial.println(" ");
  
  nextMillis+=500;
}

dcb: thanks for doing these calculations. One other thing that would be useful to check is the running total of the difference between the micros delta and 500,000 (i.e. how much does the change in micros differ from the expected change). The running total of micros deltas just says that the values are always increasing, but not necessarily that they do so by the right amount each time. Also, I'm wondering what the minimum increment (i.e. best resolution) you can get from the micros() function is. Is it on the order of 1 us, or more like 10? 100?

Also, I'm worried that if the values returned from micros() don't overflow on a regular data size boundary (e.g. 2^16 or 2^32), they will be difficult to work with. What if keep a running count of the micros() inside the timer0 overflow as we do with millis now? Or can we somehow truncate to, say, an unsigned int so that the overflow works out properly?

Also, I'm wondering what the minimum increment (i.e. best resolution) you can get from the micros() function is.

Even ignoring the amount of time the micros() function call itself takes, the resolution would be capped at 4us (8us on the 8MHz processor) for the simple reason that a single tick of TCNT0 takes 64 * 1/16 =4us.

Mikal