You said what it should do, but neglected to say what it does do which would be helpful.
if (buttonState == HIGH) {
// turn LED on:
digitalWrite(ledPin, HIGH);
} else {
// turn LED off:
digitalWrite(ledPin, LOW);
}
That code says that if the button is not pressed, turn the LED on and when the button is pressed turn the LED off. The wiring of the button makes the input HIGH when not pressed and LOW when the button is pressed.
You need to swap the logic in the if statements.