What's a good place to start re: equipment (supplies, kits, etc...), and what's the usual no 1 online supplier that you guys could refer me to?
First off, learn to shop surplus; you can get great deals that way. My favorite places:
http://www.allelectronics.com/index.php
Between the three, they should have everything you need to get started. If you are just starting out with the Arduino, you mainly want to get a breadboard or three as already been suggested, plus some basic components, plus some jumper wires. If you have the cash, you could spring for the nice jumper wire sets out there (some like the stiff, pre-formed wires that come in a little plastic case, others prefer the looser bendable jumpers that you find on ebay and elsewhere), or if you want to save a ton of cash, find a place that sells 24-pair telephone cable, and order 3-6 feet of it, then cut the jumpers yourself.
You'll want some basic tools if you don't have them (screwdrivers, pliers, wire cutters, forceps, etc). If you are just starting out, and not figuring on soldering anything soon, you can hold off on purchasing a solder iron or soldering stations. Definitely get a multimeter or two (also, if you are serious about this hobby - buy an analog meter as well; it can be helpful in certain situations where a digital meter isn't, unless you spend a ton of money on the meter).
Don't worry about items like oscilloscopes, frequency counters, frequency generators, etc; these are tools that are helpful for certain project troubleshooting, but for basic learning they aren't needed, and you can spend a fortune on them and then find you don't use them as often as you thought.
For your parts, look at the components included in the various kits; build up a list of those parts and values, then go shopping surplus for as much as possible. Another place I would look into is Earthshine Electronics:
http://www.earthshineelectronics.com/
They are based in the UK, but they have an awesome kit and book (the book is a free PDF download, btw); well worth looking into. Other places for good learning information on a variety of general electronics topics:
http://www.kpsec.freeuk.com/
http://www.electro-tech-online.com/ (for some reason, it is down today...)
Also - get a subscription to Nuts and Volts Magazine, and if you are into robotics, their sister publication, Servo Magazine:
Finally - if you are really serious about learning electronics, I highly recommend you to pick up a copy of Grob's book "Basic Electronics"; this is a textbook used by many EE101 type courses; it starts with the bare basics ("What is an electron?") and moves from there, covering the theory and practice of using electronics for circuit design and troubleshooting. Pick up an older edition if you want to get it cheap, since it is a textbook - current editions can be pricey; any edition published in the last 5-10 years will work.
Finally, note that there are a ton more than the three surplus places I mentioned out there; some of the places specialize in certain kinds or categories of components. You don't mention where you live, nor where you interests in electronics or using the Arduino lay - knowing this info could help us find you places to check out...
