There is a delay(20) in heartbeat(). This is what slowing the sketch down.
The solution to this problem is get rid of the delay() and re-implement the heartbeat using millis(). My M184v1 Arduino-based AVR ISP is now shipping with this slightly modified ArduinoISP sketch.