GPS is noisy. Mucks up serial.

I've built a few projects with GPS. In general I've found that it's hard to upload a sketch when the GPS is connected to my arduino and powered. If I disconnect I can flash the sketch, then re-connect and it will work.

I connect to TX/RX. Is this related to the serial connection of an arduino to my computer?

Also, I'm trying to integrate an MP3 player shield with GPS. The sketch I'm using lets one pick songs in the serial terminal. When I add the GPS the serial terminal goes nuts. It's like it's getting random commands.

What is going on?

I connect to TX/RX. Is this related to the serial connection of an arduino to my computer

Those are the hardware serial pins. When you upload code, it is going to the GPS, too. Probably not too useful, as far as the GPS is concerned.

Also, I'm trying to integrate an MP3 player shield with GPS.

How are you connecting the MP3 player to the GPS? Does the GPS know how to deal with the MP3 player?

When I add the GPS the serial terminal goes nuts. It's like it's getting random commands.

You are trying to use the one serial port for two different purposes. It's not surprising that results are not spectacular.

Yes, it's related to the connection to your PC, pins 0 & 1 are already connected with your PC and connecting another serial device will give problems, it may even ruin components.

Unfortunately atmega328-based controllers have 1 (hardware) serial port, but you can use 2 other pins and create a software serial port as well.

https://www.arduino.cc/en/Reference/SoftwareSerial