The led should follow the state of the button. Perhaps the input is floating, check to make sure that the switch is connected to pin 7. If you have a voltmeter, check that the voltage is 5 volts on pin 7 when the switch is not pressed and 0 volts when pressed.
Also, are you sure your sketch has been sent to the board, as I recall on mine. the default sketch just flashes the led.