Yes, that is what is so truly terrible about the "tutorials" on the Arduino site which "teach" things entirely contrary to good engineering practice.
More practical and generally safer. Wiring the power supply line to switches, contacts or in fact anything that you do not need to, carries a risk however slight, of interference with the supply either from picking up impulses from other devices or mechanical damage and being shorted to ground.
In addition, the microcontroller provides a built-in pull-up resistor function of approximately 45k, which may be sufficient to not require an external component.
Any, preferably small, a reed relay would probably be very appropriate.
It suppresses a substantial voltage surge when the transistor switches off the current to the coil, called "kickback" and other things.
Thanks Paul and Mike. I am truly learning a lot here.
So from what Mike previously suggested, I would have to wire the button as follows (the right 5V wire goes to the relay so ignore it for the sake of the button discussion) :