I have already solder everything together so removing everything and adding them back one by one doesn't seem like a preferable choice. 
You can make, as a test, a "bare bones" board, even on a breadboard. Also there are cheap circuit boards, for example this one for $US 3:
http://evilmadscience.com/productsmenu/tinykitlist/74-atmegaxx8
Then solder on a chip header (eg. the ZIF socket they have on the page), a couple of decoupling caps, a resonator, and then the wires leading to your circuit.
Then you are in a position to test the sketch/hardware combination. You can power it, for testing, from an external 5V supply. For example, I powered my test board by simply connecting to the 5V/Gnd pins on an Arduino Uno, which supplies me with regulated 5V.