This is why the usual advice is to avoid delay(...) and use millis() instead. However, you may have to rearrange your program.
delay(...) prevents most useful things from happening. In software circles, this is known as blocking. millis() is nonblocking.
Read the setting of the switch.
Use millis() to find out the current time.
Use subtraction (and ONLY subtraction!) to determine whether sufficient time has passed.
(Why do I say to use subtraction? It is because millis() returns an unsigned long that will roll over after 49 and a fraction days. Subtraction is guaranteed to work in this case, but not addition. See
if you want more information.)