That has been my experience as well.
I built a three-minute timer for my wife that runs directly from two AA batteries using the internal oscillator (ATtiny13). Over the entire voltage range (3.2 V to 1.6V) there is less than a two second difference (2/(3*60)*100 = 1.1%).
This should give you a rough idea of what Atmel guarantees...