When I change it to 57600 (or anything else) I keep getting errors. e.g.
If you look at the source of ArduinoISP, there is a call to a blink led function called "heartbeat":
// light the heartbeat LED
heartbeat();
if (Serial.available()) {
avrisp();
}
If you comment out this call to heartbeat, you should be able to run ArduinoISP at 57600 baud.
The problem is that heartbeat uses delay and while blinking at high baudrates, serial input data will overflow the input buffer and loss of data occurs. Baudrate is a limiting factor and so increasing it should give you a faster overall programming speed as well.
WARNING!
Please note that when changing the baudrate to 57600 there is no protection against incidentally invoking the Arduino bootloader rather than the ISP sketch it is running. As a consequence you may erase and reprogram your programmer (Arduino) rather then the target device.
Unless you are aware of this issue and understand how to safely disable auto-reset, I would recomend keeping speed at 19200 baud and not attempt this modification.