Yes Tom, you just need the push switch wired to the input pin (in my code this is set to pin 3 and I call it ButtonPin) and to Gnd (ground or zero volts).
The INPUT_PULLUP causes the pin (pin 3) to normally be set to high via an internal pullup resistor and the reason for this is that it makes it more stable because if you don't do this and you use electric motors then it can spontaneously change with the electronic noise from the motors.
Here's the tutorial about INPUT_PULLUP
http://arduino.cc/en/Tutorial/InputPullupSerialCheers
ian