I got my pushbutton from a pack from NKC electronics for the Getting Started with Arduino book... it didn't stick very well in the breadboard they sent me, but that's ok because the breadboard was tiny anyway and I quickly needed a larger board, it sticks fine using it across the median gap
http://www.amazon.com/gp/product/B001M0KGME/ref=oss_product - the larger breadboard
http://www.amazon.com/gp/product/B002VH51OU/ref=oss_product - book pack
As for popping out? I just leave it in the board, hasn't come out yet, except for when I was first using it, I was putting it in sideways (which kind of ruins the point of a button in the first place :P)