Unwanted behaviour when using millis to prevent button debounce

The following code works alright, but if I continue to press the button on pin 2, the LED on pin 13 starts blinking. This is an unwanted behavior; I don't want any blinking. Can someone explain to me why this is? In my view, it should be impossible to enter the if statement if the button is long pressed.

const int buttonPin = 2;     // the number of the pushbutton pin
const int ledPin =  13;      // the number of the LED pin
int ledState = LOW;
int buttonState;         



unsigned long lastDebounceTime = 0;
unsigned long debounce = 200;

void setup() {
  Serial.begin(9600);
  // initialize the LED pin as an output:
  pinMode(ledPin, OUTPUT);
  // initialize the pushbutton pin as an input:
  pinMode(buttonPin, INPUT);
}

void loop() {

  buttonState = digitalRead(buttonPin);

  if (buttonState == HIGH && millis() - lastDebounceTime >= debounce) {

    lastDebounceTime = millis();

    ledState = !ledState;

    }   
  digitalWrite(ledPin, ledState);

}

You need to only react to changes in the button state after debounce. Something like:

#define DEBOUNCE 200

void loop()
{
  static unsigned long lastDebounceTime = 0L;  // private static variables.
  static bool buttonState = false ;
  if (digitalRead (buttonPin))
  {
    if (!buttonState && millis() - lastDebounceTime >= DEBOUNCE)
    {
      buttonState = true ;
      digitalWrite (ledPin, !digitalRead (ledPin)) ;
    }
  }
  else
  {
    buttonState = false ;
  }
}

buttonState is now the debounced state of the button, and we only toggle the LED when buttonState transitions to true. I used a define for DEBOUNCE to avoid paying the price of 4 bytes of SRAM for a long variable. You could alternatively declare it const.

Your code assumes a button wired between the pin and Vcc with a pull-down resistor to ground - is that what your circuit has?

did you use the external pull-up or pull-down resistor?
If not, you could use an internal pull-up resistor. See this post Button FAQ: common mistake - button does NOT work as expected. - Introductory Tutorials - Arduino Forum