you might have trouble with the bootloader/serial monitor, etc. or with anything else time-related
The 8MHz internal RC oscillator is calibrated at the factory and in my experience, is almost always accurate enough for serial communications, at least up to 56 KBaud. I have had a few failures that might be attributed to timing errors, but I was never certain of the cause.