So I'm guessing that while the switch is open the pin in is held low at 0V.
Without the resistor, the pin is considered to be floating and would be susceptible to problems with electrical noise. Right?
You've got it.
A couple of other notes: more often than not, lines are pulled "up" to 5V rather than "down" to ground. This has the effect of making your switch logic inverted (i.e. pin is low/0 when button is pressed, high/1 otherwise).
The Arduino's ATmega has some built-in pullup resistors that can be enabled, to save you the trouble of adding them to your circuit. I don't recall for sure off the top of my head how to enable them - IIRC, you set it as an input, then write a 1 to it as if it were an output.
-j