That's no problem at all!
With the switch on and the input pulled to ground, there is a small current through the internal pull-up resistor.
Of course if the switch is on 90% of the time, you can reverse your logic and make "off" the "normal" condition. But as far as reliability, it doesn't matter.