If you buy a starter kit, buy the official Arduino starter kit. There are starter kits from Ebay that are disappointing.
You can also select parts from https://www.adafruit.com/
If you want to be cheap, you can buy an Arduino clone from Ebay, but it will have the cheapest components and you are not supporting the Arduino team.
What is your interest ? sensors ? or led lights ? or servo motors ?
The Arduino is to learn an try things. Start with an Arduino Uno board, and make a led blink. You continue with sensors and a LCD display and so on.