after a reset its normal to have a few seconds waiting time but if you don't reset it manually (press the resetbutton, upload software or power it of and on) it shouldn't happen ... hmm, what led? the onboard one on pin 13? if you use another led it can be a simple connection problem or you made a mistake with your programming (randomnumbers or something like that)
Put them on a picture hosting site like Flicker and get the URL.
Then paste that URL between the brackets you get when you click the picture icon (third from the left) in the reply box.
Just a thought but have checked the board for loose components or even a dry joint.
Or perhaps there is too much electrical noise which is stalling the crystal osillator or even dust close to the oscillator affecting its load capacitance.
I would suspect that some software on your computer is periodically trying to open the serial port that it finds due to the arduino. Because of the auto-reset feature, this will cause the Arduino to reset, appearing as in otherwise inexplicable pause.
You can check for sure by something more noticable at the very beginning of your sketch (blinking the on-board LED a couple times rapidly, for instance.)
This wouldn't be completely unlikely behavior for cell phone or PDA sync software...
Lol I once had this problem with a big ball of dust which accumulated on the PCB near a high frequency section, it could have had a metallic residue in it.