Other hardware depends on what you want to do with it.
Motors? Need motor shield.
LEDs, LED matrix? MAX7219.
0.1uF caps, 1uF and 10uF electrolytic caps, 270 ohm resisitors, solderless breadboard, jumper wires.
Shift registers always handy - 74HC595 for low power outputs, TPIC6B595 for high current sinking (great for LEDs). 74HC165 for digital inputs.
Buttons/switches, 10K pullup resistors.
10K potentiometers for variable analog input.
5V wallwart, 7.5-9-12V if need higher for motors.
Poke around here, www.dipmicro.com, lot of common parts can be located easily.