It depends really whether you are learning electronics or Arduino programming.
If its primarily attaching electronics to an Arduino, I like the Oomlout kits (http://oomlout.co.uk/) although they are in the UK.
To learn to program the Arduino, if you have a bit of programming knowledge, then see what Amazon recommends for Arduino books. If you have very little programming experience, then I think I think my 'Programming Arduino: Getting started with sketches' is one of the few that assumes no programming experience.
If you want a definitive bible, I think the 'Arduino Cookbook' is pretty good.
I second the recommendation for 'Programming Arduino: Getting Started With Sketches' I just picked this book up and have learned an incredible amount from it.