I've been suspecting my often flaky USBtinyISP of causing me headaches with a project. But today it would communicate with the processor for about 15 seconds after connecting it and then I'd get the "initialization failed" message until I unplugged everything and let it sit for a while. What could that mean?
I eventually figured out that the CS pin of the lone SPI device was dropping to ~1V when reset was applied. The device has an integral 70kΩ pull-up resistor. Could it have been damaged somehow? That seemed unlikely.
Long story short, it was dirt. Some sort of goo, possibly remnants of the adhesive on some electrical tape I used to insulate the board from nearby components. You can see it in this photo: a thin strand of black gunk bridging between the ground pin (top) and the CS pin next to it. It was like a pull-down resistor.
I guess the electrical tape was a bad idea.