Why does LED turn on w/o button pushed? (Absolute beginners question)

Hi, I have a very simple program where a button turns on a LED while the status of the button is shown on the serial monitor.

Sometimes the LED lights up even without pushing the button - apparently randomly. Why does this happen?

/*
  Digital Button Read
 */

int pushButton = 2;
int ledPin = 13;

void setup() {
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);
  pinMode(pushButton, INPUT);
  digitalWrite(pushButton, LOW);
}

void loop() {
  digitalWrite (ledPin, LOW);
  int buttonState = digitalRead(pushButton);
  Serial.println(buttonState);
  if (buttonState == 1)
    {digitalWrite (ledPin, HIGH);
    }
  else {digitalWrite (ledPin, LOW);
  }
  delay(100);
}

How is the pushbutton connected to D2? Is it connected to GND or 5V? Does it have a pullup or pulldown resistor?

Change this line pinMode(pushButton, INPUT); to pinMode(pushButton, INPUT_PULLUP); // input with internal pullup resistor

and change this if (buttonState == 1) to if (buttonState == 0)

and wire the button to connect the pin to Gnd when pressed.

Then the pin will read 1 when the button is not pressed due to the internal pullup, and 0 when the button is pressed. Much more stable results that way as the pin is not acting as an antenna and being affected by random electric fields.

and wire the button to connect the pin to Gnd when pressed.

Many of the common 4 leg button switches can be confusing to wire in that some terminals are always connected to each other and are not switched.

If you bring the wires from diagonally opposed corners of the switch, one wire to ground, and one wire to the input pin, it will always work as intended.