Have a search across various Arduino supplier sites- they tend to have pre-packed sets of odds and ends.
Lots of vendors, such as Sparkfun, Adafruit, etc., will have "experimenters" kits, or fun project kits. Check out things such as parts grab-bags, etc., such as at Yourduino
. If you want to play with motors, Adafruit has a motor shield kit
. With a lot of surfing, you can find things more cheaply than buying the kits from Sparkfun or Adafruit, but until you have learned a bit, the kits can be easier than buying parts from Mouser, Dip Micro, Digikey, etc. In some cases, the kits could end up being cheaper, especially if buying from Asia.