Why does the LED turn on after the while loop becomes false?

no. you'll need to test for that value and reset it or use the mod operator

ledMode = ++ledMode % 8;

it's odd the way this code is conditional on the button being pressed (== 0) and then waits for it to go HIGH.

doing things in conventional ways makes it easier to recognize what the code is doing and recognizing bugs