I have a simple project that uses an 8MHz system clock. It has a seven-segment display that is multiplexed directly by the '328P. One of the timers is set to interrupt every 100µs (10kHz interrupt frequency), and the ISR actually manages the display. Now the rest of this project is doing essentially nothing from a CPU load standpoint, just human input via a few buttons, and that is certainly part of the equation as RC noted. But it works great, response to the buttons seems immediate.