Can I use the AVR Pocket Programmer with the Arduino Mini 05?

Nope. The Programmer does not speak TTL serial (At least in that way).

You could hook it up normally (MOSI->MOSI, MISO->MISO, SCK->SCK, etc) and upload a sketch and over-write the bootloader. If you want to put the bootloader back on, you will have to re-burn it.