The first example really does generate a non-square wave. It ought to be obvious: 5.33 = 16/3, so it's a 3-cycle waveform. Lacking any sort of clock multiplier, that's going to be two cycles in one state and one in the other state... (or, you're counting from 0 to 2 and flipping the bit at 1, so depending on which edges are involved, we have one state for 0,1 and the other for 2, or one state for 0, and the other for 1,2...)
The second example did generate a 8Mhz square wave (on pin 10 of the diecimila, I think. I didn't check both 9 and 10.) Interpret the code as "count from 0 to 0 and toggle the output each time you finish (ie toggle every cycle, so output = clock/w))
The timer-generated output signals are quite a bit less interesting than the software-controlled ones, IMO. Although people seem to have used such things to help drive LCD displays in a jitter-free fashion... Hmm.