I'm using an Arduino Uno, which I've programmed as a bike computer. I wanted to add a button to pause the LCD so data can be collected from the screen.
I put two pins of the button on a 5V rail (on a protoshield) and connected one of the others to digital pin 10. I think programmed the button just to control the LED on the board. When I turn the board on, the light is always on without me even touching the button. I tested continuity and pin 10 shouldn't be getting any power at all but it is.
When nothing is connected to an input that pin can read high or low at any time. This is called a floating input.
To attach a button connect the button to make contact between the input pin and ground. Then in setup enable the internal pull up resistors with a pinMode call.