It's called a pull-down resistor. It's there to make the pin read LOW when it is not connected to +5V. Without it, the pin will 'float' and produce random results.
The Arduino UNO/NANO/MINI/MEGA/Leonardo/Micro have built-in pull-up resistors you can enable:
pinMode(BUTTON, INPUT_PULLUP);
If you use it, you don't need the external resistor. You, just need a button between the pin and Ground.
On almost all Arduinos, HIGH==true==1 and LOW==false==0 so your loop() can be simplified to one line:
void loop()
{
digitalWrite (LED,digitalRead (BUTTON));
}