I don't have one of those switches... thought I did :stuck_out_tongue_closed_eyes:
But if you have a look at the Fritzing pic on this tut, it looks to me like your switch is the wrong way round?
The switch joins the pins on the same side when it's pushed, not the opposite pins which is more intuitive and is the way you have yours: if I'm right then your red and yellow are always joined, not switched.
Edit.... I dug around and these switches seem to be (or are similar to) Omron B3Fs. Have a look at the pic attached from their datasheet, which indeed shows that opposite pins are always connected, and pushing the button connects the pins on the same side; in fact connects all 4 pins.