delay() doesn't work in interrupt

I know it's bad practice and I was just doing this to test that the pin interrupt is working. The LED flashes for a much shorter time (just tens of milliseconds) when the debounced button is pressed. Very curious...Why?

void setup() { 
attachInterrupt(digitalPinToInterrupt(2), timeStamp, FALLING);
pinMode(2, INPUT_PULLUP); 
pinMode(LED_BUILTIN, OUTPUT); 
}

void loop() {
  }

void timeStamp()
{
digitalWrite(LED_BUILTIN, HIGH);
delay(5000) ;
digitalWrite(LED_BUILTIN, LOW);
}

Ahh, that makes sense if delay() is also using an interrupt. Thanks.