On a Duemilanove or UNO pin 13 has a resistor and led on the board, but I added a brighter LED because the one on the board is rather dim and hard to see. The pin directly above it is ground. http://www.arduino.cc/en/Tutorial/BlinkingLED
That is bad and outdated information on that linked page. The present arduino boards do not have a resistor wired in series between the chip pin and the shield pin 13, and so if you simply wire a led like in that picture there is no current control that limits current to less then maximum safe value for the chip's output pin.
If you wire an led directly between the shield pin 13 and the shield ground pin, the led will have no series resistor to limit it's current flow and puts the 328p chip and led at great risk. The external led requires, as in you must add, a series resistor to prevent damage.
Lefty