The danger is that if you have to the pot turned all the way in one direction such that you have a 0 Ohm resistance between the +5v line and the input, if you push the button you would short the power supply.
What you can do is use only two pins of the potentiometer - the wiper to the pin, and one of the other terminals to ground. Then have a 10k resistor from the pin to VCC. That way you won't short the supply and you don't need to use INPUT_PULLUP.
However, if you have the pot turned all the way in this situation, the pin will be connected directly to ground - meaning that you would never be able to register the button press as it would be indistinguishable from the pot.
This leads to a third circuit which would allow you to use both. I have attached a schematic.
Basically, if you do analogRead() you will get a number between 636 and 988 depending on where you turn the pot. If the button is pressed, analogRead() will return 0.
If you do digitalRead() when the button is pressed you will get LOW. If you do digitalRead() when the button is not pressed, you will get HIGH - because the potentiometer will never take the voltage at the pin below 3V which is the threshold for a logic 1.
