Double check your wiring and make sure that pull-down resistor is connected to gnd and the other end to the pin on the arduino that you're using for reading the state of the button.
Make sure your switch is fully opening when not being pressed. This assumes you have a multi-meter handy so that you can read Ohms. A faulty switch could cause the problems you're seeing.