Project 02 - button does nothing

No, rotating it 180 degrees would make no difference. Rotating it 90 degrees on the other hand will make a difference. Two pairs of the legs are always connected to each other. The connection between the pairs is only made when the button is pressed. So if you put it in the breadboard at the wrong orientation it will always conduct whether the button is pressed or not.