We've all been there before..
the Arduino platform is what I like to refer to as a development platform...
it has a lot of built in features and components to make quick/fast prototyping of circuits EASY!..
In a nut shell.. the Arduino has a Atmega328P microcontroller chip at its base... these chips have some built-in functionality that we can use for our projects... the Arduino platform breaks out these pins.. to pin headers so it is easy for use to connect external compoennts to the Arduino.
You then use the Arduino IDE (software/application) that is used to communicate with the hardware,.. and allows you to upload your code (called sketches) to the board..
the IDE is located here: (latest is IDE 1.0.3)
Getting started with Windows Guide:http://arduino.cc/en/Guide/Windows
Jeremy Blum tutorials:http://www.jeremyblum.com/category/arduino-tutorials/
I would suggest getting familiar with the hardware a bit.. the terminology used (as it will help cut down on confusion when you see these same words/terms used over and over)
Common problems/errors people run into are: (IMHO)
1.) drivers for Arduino
2.) drivers for FTDI cable
3.) not selecting the correct board type in the IDE (tools >> board >>)
4.) not having the correct COM port selected (tools >> serial port >>)
I say download the software.. install everything you need to..
and hook up the Ardiuno Uno via USB cable.. (make sure settings are correct)
at this point you do not NEED any other external components to start 'playing'..
The Arduino comes (usually) with a built in LED on pin 13...
so you can stat playing with code and upload sketches right out of the box..
the normal one people start with is the default BLINK sketch..
form there analyze the code..and make a few changes.. upload the code again and see how the changes you made effect the led/Arduino..