Eliminating the bootloader makes updating the firmware a bit harder, but it also increases the amount of flash you have for your program, as well as removes the boot delay while the loader waits for escape character, letting it know there's new firmware to load via serial.
Yeah, and for reasons I'm not entirely certain of, the bootloader that I'm using through the Burn Bootloader command is the (a?) no-wait bootloader that only enters serial programming mode after a reset. It's disconcerting to use my official Arduino that has a delay on powerup now.

Of course, now that I've re-routed all of my signal lines to the top of the board, I have the entire bottom of the board (more or less) to route an ISP header to. I might just put both headers on the board, but I'd like to keep the no-wait loader I've been using because I ran out of SRAM before I ran out of flash, so the bootloader overhead is in a section of memory I'm not even touching anyway.
I am making a color mixer and fader for RGB LED lighting, and found the boot delay annoying when I powered up the system. Removing the bootloader makes the startup nearly instant.
I use a USB-ASP I got off ebay for a couple of dollars. In reality, it is just an ATMEGA8 running what appears to be a software USB stack and ASP firmware similar to using the arduino as an ISP.
That's the one I'm using too - $5 and it programs ATtinys like a champ without making me rip up my Arduino. Still waiting on my ZIF sockets to make it a little less breadboard-y, but it's a lot simpler than fiddling around with ArduinoISP.