Im having a similar situation with a custom board that has an atmega2560. the ISP port is connected to a USB-SPI chip to talk to a PC without the need of drivers (like for the FTDI) im trying to program the bootloader but this error appears:
avrdude: stk500v2_command(): command failed avrdude: stk500v2_program_enable(): bad AVRISPmkII connection status: MOSI fail, SCK fail avrdude: initialization failed, rc=-1 Double check connections and try again, or use -F to override this check.
Im using the avrisp mkii and the custom board has all the caps and osc stuff in place. could it be that this chip is interfering? thanks