Just in case anybody doubts the power of Serial.print markers in code requiring debugging, have a look at replies 6 and 7 in this thread. Reply 6 was my question, wit copy / paste from the monitor; Reply 7 was zoomkat's answer.
I used to be gung ho about source level interactive debuggers, but in recent years simple trace output has been my go-to debugging method. It has lots of advantages. There's very little difference between ordinary exception logging, and tracing an event of interest. The distinction between them is largely a matter of what problem you're trying to solve. Almost any non-trivial diagnostic logic is easier to implement in source code than in the debugger, and any diagnostic logic implemented then becomes available for future problems. The only time now that I'd prefer to trace execution in real time is where I have no idea what the code is doing or where abouts in the code I need to start looking. Even then my first thought would be to enable tracing in the relevant modules so that I can get a sense of what's happening. Best of all it carries across equally well whether I'm working on mainframes or micro-controllers and regardless of what language I'm working in or which part of the world the problem happens in.
The only time now that I'd prefer to trace execution in real time is where I have no idea what the code is doing or where abouts in the code I need to start looking.
I'm working on StateMachine stuff and thinking about having a Debug State Machine that can pop up on the Serial Monitor and tell you what other State Machines are running and what State they are/were in at last snapshot. That 'monitor' could also support debug statements in code that could be turned on and off without recompiling source. Maybe. I tend to design fantasies and then see if they can be made to work. Maybe.
I find that having two or three spare pins to pulse and read with a logic analyser does the trick most of the time.
I still keep thinking back to the days when even little machines like the 6502 exposed the address and data busses and you could make hardware tracing work. The homebrew 6502 system I built had thumbwheel switches you could set to an address and sync the oscilloscope. Guys at IBM built XY displays from the high and low 8 bits of address and drove a tektronix graphic display so you could get a "picture" of where execution was going. On a MEGA with LotsaPins I plan to hook up a Hex Display on 5 pins and be able to display 16 points of execution or values from application code. But the hardware stuff is going/gone...
But with silicon so cheap maybe even small stuff like ATMEGA could have a good built-in debugger that optionally drove 2 serial pins if the Debug Fuse was set. Maybe.
Guys at IBM built XY displays from the high and low 8 bits of address and drove a tektronix graphic display
I did that as well. I even went a step further and made an 8-ch logic analyser with a cursor and retrace blanking handled by the Z axis. Fun times.
built-in debugger that optionally drove 2 serial pins if the Debug Fuse was set. Maybe.
All done on the LPCs (and I assume other ARMs), of course there's a full debugger but you can also link in the "semihost" version of the libs and get a virtual serial port to a terminal window.
If I'm not using SPI I send stuff out that as well, makes it easy to see 8-bit values (or larger) on an analyser.
Part of the business of successful debugging is thinking about what tests will demonstrate a particular success or failure. Sometimes it may be as simple as a message to say that a specific line of code has been reached.
I reckon there are two sorts of problems
(a) silly mistakes that you make where you know how to do it correctly and
(b) new stuff that you haven't done before (such as interfacing to a new sensor or using a new library) where you may make mistakes that aren't obvious or you may not be sure what to expect.