Serial vs. ICSP?

Is it all a matter of where you send the data? I mean to say, if they're both a form of Serial, what's to stop you connecting a Serial cable correctly through the ICSP and burning the bootloader? :o