There are issues with the pull-up resistors used on those pins. In principle you can use them if you want to. In my VGA output test documented here:
Towards the bottom of the page I used pins D0, D1 and D2 for RGB signals to the monitor. So yes, it can be done. But for inputs - caveat - check the schematic for the pull-ups. I think you can probably work around them. For example, if they have pull-ups, ride with it, and make your switches ground the pins. Make sure if you do this that you don't happen to turn Serial on (eg. Serial.begin) because then you may damage the pins.