Possibly. But we normally recommend to connect an ordinary button (without an LED) from the Arduino pin to ground, with no resistor, without much concern that the pin might be accidentally configured as high.
With these buttons which have LEDs, it will be necessary to switch between input and output modes, which you could argue increases the risk. But pins default to low when set to output, so I don't see it as a significantly increased risk. I guess the danger is that a beginner thinks the pin must be set to high rather than input to switch off the LED.