Push Button Not Working

I am having difficulty using a Push Button to start a sequence of flashing LEDs. I know I have wired it correctly, as when I remove the Button, the LEDs flash in the sequence I programmed them to. I believe the problem may be with the Button itself, however, I am not sure. If it may help in solving the problem, I have included my sketch.

const int ledOne = 13;
const int ledTwo  = 12;
const int ledThree = 8;
const int buttonPin = 4;
const int delayPeriod = 1000;

int buttonState = 0;

void setup()
{
  pinMode (ledOne, OUTPUT);
  pinMode (ledTwo, OUTPUT);
  pinMode (ledThree, OUTPUT);
  pinMode (buttonPin, INPUT);
  
}

void loop()
{
  
  buttonState = digitalRead(buttonPin);
  
  if(buttonState == HIGH)
  {
    digitalWrite (ledOne, HIGH);
    delay (delayPeriod);
    digitalWrite (ledTwo, HIGH);
    delay (delayPeriod);
    digitalWrite (ledThree, HIGH);
    delay (delayPeriod);
    digitalWrite (ledOne, LOW);
    digitalWrite (ledTwo, LOW);
    digitalWrite (ledThree, LOW);
  }
}

Moderator edit:
</mark> <mark>[code]</mark> <mark>

</mark> <mark>[/code]</mark> <mark>
tags added.

Need to see your circuit. How is the button wired?

Edit: Easy enough to test a switch, use the circuit below.

ledSwitch.png

Be sure you always know the state of that button of yours.
What is the state when the button is not pushed ?
What is it when the button is pushed ?

If you enable the pullup at your input pin and connect the button to GND, then you'll have a HIGH when the button is not pushed, and a LOW when the button is pushed.
If you only have it at some level (either HIGH or LOW) when the button is pushed, what state will it be in when the button isn't pushed ?

You aren't enabling the internal pull up, so are you using an external one?

http://www.cmiyc.com/tutorials/arduino-pull-ups/

I think you have not wired the button correctly.
Pin4 -> button "+"
pull-up resistance (like 10kOhm) -> button "+"
button "-" ->GND

If you have four wires onto your push button, make sure the "+" and "-" are in the same side (so not "+" in front of "-").