The LED is shining because the button sketch is reading a digital pin which isn't connected - placing your hand nearby can cause enough stray field to affect the input. This is normal. If you connect a push button and pull-up resistor the LED should respond to the button.