simple serial issue

I've now added common ground.

I guess one can actually brick a Arduino by writing a program that constantly sends some trivial data (like "A") to the PC

void setup() {
Serial.begin(4800);
}

void loop() {
Serial.print("A");
}

That would make it impossible to ever write programs to it again, as it will hang, just as mine is doing now, when I upload while my GPS is connected.