I always advise people to take a look at the parts in any "kit" (maybe multiple different kits - make a list of parts, common ones, and such) - then shop around for those same parts on Ebay and other vendors (especially surplus, if you have them in your area). Make sure to keep in mind shipping costs (both of the kits and for your parts). Try to get as many parts from one place as you can.
Most of the time, these kits go for around $100.00 USD (plus whatever shipping/handling). If you are very careful, you can get a real Arduino for about $30.00 or so (from one of the official suppliers, like SparkFun or similar), then you can make that $70.00 left over go a really, really long way.
The first thing you should get, though, is a cheap multimeter - most kits don't have them as part of the kit (a $5.00 meter is all you'll need to start with). Then a small breadboard (go for the one with 840 tie points or so - don't go for anything smaller). For hookup wire, don't go for the jumper wires if you can avoid it; instead, get some 22 gauge hookup wire (or, if you can find it surplus - a 6 foot length of 25 pair telco cable is perfect for jumpers - you'll end up with a thousand jumpers for little money in the end). In a pinch, Cat5 cable can work - but the gauge of the wire is fairly thin...the thing is, stuff like this is thrown away during office remodels - so try to find an office building/site being remodelled, and ask the site foreman about getting some.
After that - basic parts are what you want. If you are in the States (or don't mind ordering) - purchasing one of these can be an interesting exercise:
http://www.goldmine-elec-products.com/prodinfo.asp?number=G9321
Believe me - you'll have more parts to play with than you ever thought possible. Order two just to be sure! Tons of diode, LEDs, transistors, capacitors, etc - like you wouldn't believe!
Or - go on Ebay - there are a ton of chinese electronics suppliers out there; many times you can get grab-bag or "kits" of parts from them as well.
Finally - whatever you decide to do - figure out a good way to store and sort all of these parts (especially in the case of that Super Surprise Box I mentioned - you really won't believe it until you see it)...