The code will read the present state of the input at the time you read - it doesn't rely on watching for a change (unless you want it to) so if a switch is already set on power up it will be fine. The time from power on to getting to the first pin state read will be measured in milliseconds. If this is a push-button switch I'm expecting that will be fine as they don't change that quickly.
You're right to point out the linear regulator but that can be bypassed by feeding 5V from a switch mode supply, battery etc into the 5V pin directly.
Will it be just milliseconds? Or does it not have to wait for the bootloader to 'time-out' after waiting a fixed amount of time to see if there is an active upload request coming from the IDE?
Bootloader behaviour on how long it waits has changed over the years depending on the specific bootloader version installed on the chip. Seems to me the some older bootloader designs always waited a fixed period (some seconds? checking for a upload request) no matter if the bootloader started via a hardware reset or a power-up state. Seems that (some?) new bootloaders do the IDE upload check only after a reset condition, but if it was a power up condition it does a 'no wait' and will immediately jump to the users code (assuming there was one burned to flash from a prior upload to jump to).
Just asking, as the now pretty long legacy of Arduino hardware and software can create some rather confusing situations for users depending on the age of their boards.