There is more information about how to connect the LED here: http://www.arduino.cc/en/Tutorial/BlinkingLED Basically you just put the long lead of the LED in pin13 and the shorter lead in the hole labeled GND. The Arduino board will supply the power on pin 13.
For wiring things up easily, breadboards are essential. Look here for info: Breadboard - Wikipedia