When an input is not connected to anything it is known as "floating". This is when there is no signal on it, so the input cannot know what logic level it is at.
When you connect the input (via a button, say) to either Vcc or GND, you are placing the input into either the HIGH or LOW logic state.
You want to avoid a floating input at all costs. They can cause strange results in programs. To avoid them you place the input into a "default" state - the opposite of the way you want your button to connect the input to Vcc/GND. For instance, if your button connects the input to Vcc (HIGH), then you need to set a default state of LOW. So you need to connect the input to GND for when the button isn't pressed.
However, you can't just connect it straight to ground, because then when you press the button there will be a direct connection between Vcc and GND, and that will basically be a short circuit. The whole system will die, and in extreme cases batteries will blow up and wires / traces will melt.
So, you create the default state through a resistor. The resistor can be quite high as you don't want much current to flow through it when you press the button, but at the same time it needs to be low enough that not too much voltage is dropped across it. 10K? is a typical value.
Now, when the button isn't pressed, the pin is connected to GND through the resistor and reads LOW. You press the button, and the resistor is connected between Vcc and GND, and the input is connected to Vcc, so it reads HIGH. No more floating.