Remembering button state

Your whole logic is wrong.

What you want is a sequence more like this:

  1. Turn on green light.
  2. Wait for button to be pressed.
  3. Turn off green, turn on yellow, delay.
  4. Turn off yellow, turn on red, delay.
  5. Turn on crosswalk, delay.
  6. Turn off crosswalk, delay.
  7. Turn on yellow, delay.
  8. Turn off red & yellow, turn on green. Go back to 2.

or whatever sequence your country uses.

I'd write it something like (untested) this:

void setup()
{
  pinMode(green,OUTPUT);
  pinMode(yellow,OUTPUT);
  pinMode(red,OUTPUT);
  pinMode(crosswalk,OUTPUT);
  pinMode(button,INPUT);
  digitalWrite(green,HIGH);
  digitalWrite(yellow,LOW);
  digitalWrite(red,LOW);
  digitalWrite(crosswalk,LOW);
}

void loop()
{
  // Wait for button press
  while(digitalRead(button)==LOW);

  // Yellow on, green off
  digitalWrite(yellow,HIGH);
  digitalWrite(green,LOW);
  delay(1000);

  // Red on, yellow off
  digitalWrite(red,HIGH);
  digitalWrite(yellow,LOW);
  delay(1000);

  // Crosswalk on
  digitalWrite(crosswalk,HIGH);
  delay(5000);

  // Crosswalk off
  digitalWrite(crosswalk,LOW);
  delay(1000);

  // Yellow on
  digitalWrite(yellow,HIGH);
  delay(1000);

  // Green on, yellow off, red off.
  digitalWrite(green,HIGH);
  digitalWrite(yellow,LOW);
  digitalWrite(red,LOW);

  // loop() runs again.
}