The LED is connected like this:
OK, unpower your Arduino and don't use that led wired like that until you install a series resistor of the size we have already calculated or higher, 75 to 500 ohms should work. The way you have it is shorting out pin 13 and it will soon burn out and may already be weakened and fail later. Don't be too concerned as replacement AVR328 controller chip with bootloader installed can be had for around $6: ATmega328 with Arduino Optiboot (Uno) - DEV-10524 - SparkFun Electronics.
PS: Order a multimeter while you are at it. 
Lefty