Uploading to ATmega328 from Raspberry Pi

Yup, the resonator effects all the timers, while you may be able to use a slow serial protocol, you are having issues with any reasonable serial speed. This is a known issue.
Why, if I may ask, did you choose the resonator and which model is it?