Buying a debuger

If you have windows, you can install AVRStudio and run/debug your sketches in the simulator.
"real" Debugging of the AVR "in circuit" requires more complex hardware (jtag or debugWire) which the arduino hardware doesn't support (I guess that IN THEORY, it could support debugWire...)