There is an internal RC oscillator option. Start with section 13.2 of the [u]ATmega Datasheet[/u]. There's more information about accuracy & calibration in section 32.4.
I've never used it and I don't know much about it. But as a rule, you need an accurate clock for serial communications (including USB), so you might have trouble with the bootloader/serial monitor, etc. or with anything else time-related.