Board only works with proto shield on.

This is kind of weird...I wired up a HD44780 LCD, and the arduino just never turned on. I disconnected everything and the PTC fuse is getting really hot telling me there's a short somewhere on the board. I removed the atmega chip and plugged it back in. Still shorting out somewhere. I decided to plug in my proto board for some reason and when I plugged it in, the LEDs came on. I removed the proto board, put the chip back in, plugged it in and nothing. I put the proto board back on and it started to work again. I was even able to upload sketches as long as I had the proto board on. This makes me think something is shorted out in the headers, but I can't see anything.

Any ideas?