...and when you're a cheapskate like me, you shop on Ebay, or surplus:
For other stuff, I also shop Goodwill and other "thrift" stores; sometimes you can find interesting stuff at "dollar stores" too.
A couple good places for servos and other R/C components:
...and for Arduino related stuff:
http://www.seeedstudio.com/depot/
http://store.nkcelectronics.com/
These are only -the tip of the iceberg- when it comes to the number of vendor links I have (and this just for electronics - I have a ton more for "mechanical" components)...