As you are on a tight budget I would recommend getting one or two of each of these:
LEDs: http://www.ebay.com/itm/100pcs-5mm-R-G-B-W-Y-ultra-bright-LED-kit-/220988641549?pt=LH_DefaultDomain_0&hash=item3373f3110d
Resistors: http://www.ebay.com/itm/New-400Pcs-1-4w-Resistance-1-Metal-Film-Resistor-Kit-Set-20-Kinds-Each-20-/390519047951?pt=AU_B_I_Electrical_Test_Equipment&hash=item5aecbfc70f
That would give you a lot of LEDs and resistors to start your learning experience.
/Henning