By "default AVR mode" do you mean the default fuse settings? Good question...officially, you're supposed to change the fuses depending on the clock source, but I'm wondering if it just might work to leave the fuses the way they are. I think it's worth an experiment. The worst that could happen is you damage the clock circuitry on the Arduino and a) it fails to work at all, or b) it will never work again using a crystal. Either way, with a DIP microcontroller that can be swapped out it's a pretty inexpensive experiment.
Personally, I'd buy a $30 AVR-ISP and have something that is known to work

I use my AVR-ISP so often that the $30 has paid for itself many times over.
--
The
Flexible MIDI Shield: MIDI IN/OUT, stacking headers, your choice of I/O pins