Related thread: https://forum.arduino.cc/t/rotary-encoder-increment-decrement/
If you're trying to implement what's outlined in this post: Rotary encoder increment/decrement you're making it more difficult.
It would work better, and the idea was, to toggle select outside the function then only call the function if select is in the 'adjust times' mode