How fast can you blink an LED (Theoretical fastest speed)

you did not mention which Arduino.

1ms is peanuts compared to that though, look at the blink without delay sketch and use micros() instead of millis() so that you get a better grasp at what's going on. using pin registers will be faster than digitalWrite()

On a typical 328P you can use the PIND register to flip a pin. if you put that in a while(true) loop, it will give high frequency (almost 8 MHz)