Hmm, I once tried to bootload an ATmega328 on a breadboard and had forgotten to connect both caps to ground. I tried several times to burn the bootloader to no avail. Until I realised my mistake, connected the caps to ground and then it worked. I have read a couple of crystal datasheets and they all required caps. I don't think a crystal will work reliably with no caps. Try to do some high speed serial communication and see if it still works...