Trying to load bootloader for 2560, nothing is working

I have a custom 2560 based board with an 8u2 usb chip with ICSP headers on both chips. There's no bootloader on it and I'm trying everything without luck. And I mean everything.

I have a Pololu AVR ISP programmer, a Arduino 2560 Mega V2 and a OSEPP 2560 (FTDI). Tried using all three to program the custom board on two different computers. I have two of the custom boards and have tried both of them also, same results.

Tried all three programmers on Ubuntu 11.10 and on Windows 7-64. I've tried avrdude, Arduino 1.0 IDE and AVR Studio 4 and 6 with all three devices to try and load the bootloader.

AVR Studio 4 can't find any of the programmers except the Pololu and when it does it says I need to downgrade it. I say ok and then it says it can't find a programmer.

Windows 7. Arduino 1.0 Arduino. Arduino as ISP, Burn firmware
avrdude: stk500_getsync(): not in sync: resp=0x00

Same error when I try to program either ICSP ports. 

Windows 7. 2560 as ISP v1. Same error when I try to program either of the ICSP connectors on the new board. 
c:\Program Files (x86)\arduino-1.0\hardware\arduino\bootloaders\stk500v2>avrdude
-p at90usb82 -F -P com3 -c avrisp -v -e -U flash:w:MEGA-dfu_and_usbserial_combi
ned.hex -U lfuse:w:0xFF:m -U hfuse:w:0xD9:m -U efuse:w:0xF4:m -U lock:w:0x0F:m

avrdude: Version 5.10, compiled on Jan 19 2010 at 10:45:23
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2009 Joerg Wunsch

         System wide configuration file is "C:\WinAVR-20100110\bin\avrdude.conf"


         Using Port                    : com3
         Using Programmer              : avrisp
avrdude: stk500_getsync(): not in sync: resp=0x00

avrdude done.  Thank you.


c:\Program Files (x86)\arduino-1.0\hardware\arduino\bootloaders\stk500v2>

Windows 7. Pololu ISP programmer. 
c:\Program Files (x86)\arduino-1.0\hardware\arduino\bootloaders\stk500v2>avrdude
-p at90usb82 -F -P com7 -c avrispv2 -v -e -U flash:w:MEGA-dfu_and_usbserial_com
bined.hex -U lfuse:w:0xFF:m -U hfuse:w:0xD9:m -U efuse:w:0xF4:m -U lock:w:0x0F:m


avrdude: Version 5.10, compiled on Jan 19 2010 at 10:45:23
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2009 Joerg Wunsch

         System wide configuration file is "C:\WinAVR-20100110\bin\avrdude.conf"


         Using Port                    : com7
         Using Programmer              : avrispv2
         AVR Part                      : AT90USB82
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PC6
         RESET disposition             : possible i/o
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page
      Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  Max
W   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ---
-- ---------
           eeprom        65    20     4    0 no        512    4    128  9000  90
00 0x00 0x00
           flash         65     6   128    0 yes      8192  128     64  4500  45
00 0x00 0x00
           lfuse          0     0     0    0 no          1    0      0  9000  90
00 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  9000  90
00 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  9000  90
00 0x00 0x00
           lock           0     0     0    0 no          1    0      0  9000  90
00 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0
0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0
0 0x00 0x00

         Programmer Type : STK500V2
         Description     : Atmel AVR ISP V2
         Programmer Model: AVRISP
         Hardware Version: 15
         Firmware Version Master : 2.10
         Vtarget         : 0.0 V
         SCK period      : 39.1 us

avrdude: stk500v2_command(): command failed
avrdude: stk500v2_command(): unknown status 0xc9
avrdude: stk500v2_program_enable(): cannot get connection status
avrdude: initialization failed, rc=-1
avrdude: AVR device initialized and ready to accept instructions
avrdude: Device signature = 0x000000
avrdude: Yikes!  Invalid device signature.
avrdude: Expected signature for AT90USB82 is 1E 93 82

avrdude done.  Thank you.

Windows 7, Arduino 2560 Mega v2, 6 pin cable from ICSP on the ATMEGA2560 on both boards. Selected Burn bootloader from Arduino 1.0 IDE as Arduino as ISP.

TX light on the 2560 Mega is flashing. There's no RX/TX LEDs on the custom board so I'm not sure if it's receiving.

Been going for 4 minutes now, and the IDE still says Burning Bootloader to I/O Board. I waited 40 minutes before with no results and has to kill the program.

I have a bootloader uploader here:

It doesn't use avrdude, so any problems you may be having with that won't apply. It took 3 seconds to upload the bootloader onto a Mega2560.

(many programmers will have trouble putting the bootloader onto a 2560, since it resides at an address beyond the 64k-word limit. ArduinoISP won't do it, for instance. USB TinyISP doesn't do it. I can't say about the Pololu programmer, but it wouldn't surprise me!

The bootloader uploader I posted above was specifically designed and tested for the Mega2560. A bit of fiddling around with "page" bytes, but it does work.

Thanks for the replies everyone.

Nick I downloaded and compiled your bootloader in Arduino 1.0.
I'm getting an error message. It's not letting me copy and paste so I'll short hand it.

md5.c:40: multiple definition of 'md5_starts'
" : first defined here
md5_process, md5_update, md5_finish all give the same error message.

What programmers do support the 2560? I may be programming a bunch of these boards and I want to make it easy.

I just redownloaded from:

http://gammon.com.au/Arduino/Atmega_Board_Programmer.zip

Compiled with 1.0 with no errors. However since I removed the md5 stuff from the project (due to space reasons) delete these two tabs from within the IDE and try again:

md5.h
md5.c

It compiled and uploaded fine when I removed the md5. I'm looking at the board, it's definitely a 2560. Your program thinks it's a 2561.

Atmega chip programmer.
Written by Nick Gammon.
Entered programming mode OK.
Signature = 0x1E 0x98 0x02
Processor = ATmega2561
Flash memory size = 262144 bytes.
LFuse = 0x62
HFuse = 0x99
EFuse = 0xFF
Lock byte = 0xFF
Clock calibration = 0xA1
No bootloader support for this device.

I hit C twice more and it said 2560. I pressed G and then this happened.

Shortened:

Committing page starting at 0x1FF00
Committing page starting at 0x1FF80
Written.
Verifying ...
Verification error at address 3E000. Got: 0xFF  Expected: 0x0D 
Verification error at address 3E001. Got: 0xFF  Expected: 0x94 
5537 verification error(s).
First 100 shown.

5537 verification errors? That's a lot!

Well, what speed is your board running at? Do you have decoupling capacitors in the appropriate places?

In the sketch find the line (in setup):

  // slow down SPI for benefit of slower processors like the Attiny
  SPI.setClockDivider (SPI_CLOCK_DIV64);

Change to a slower clock rate:

  // slow down SPI for benefit of slower processors like the Attiny
  SPI.setClockDivider (SPI_CLOCK_DIV128);

Try again. The verification errors, and the part number error, would appear to indicate a communicating problem.

If you still get verification errors can you post the initial output, the fuses etc?

Fingers crossed.

Committing page starting at 0x1FF80
Written.
Verifying ...
No errors found.
Writing fuses ...
LFuse = 0xFF 
HFuse = 0xD8 
EFuse = 0xFD 
Lock byte = 0xEF 
Clock calibration = 0xA1 
Done.

If the previous post was accurate you had the CKDIV8 fuse set (divide clock by 8) which would account for the fact that it didn't program too well at the default speed.

I was kind of worried about this. When I plug it into Windows I get an unknown device ATmega8U2 DFU. Do you have a program for that chip?

The Pololu programmer or a 2560 ArduinoISP should be able to program it, right?

Verifying ...
No errors found.

Looks like you have a bootloader. Now is the 8U2 chip already programmed?

When I plug it into Windows I get an unknown device ATmega8U2 DFU.

Looks like not.

I haven't programmed one of them (fear mainly being the reason) but with the smaller address space it should be easier. Isn't there a "flip" program or something to reprogram them?

The appropriate hex file is available somewhere, I can never remember where.

Flip isn't working for me. Says it can't open USB device.

OK, well if you stick your Pololu onto the ICSP pins for the 8U2, and find the .hex file, and upload that, it should work. Fuses might need fiddling, not sure about that.

Here:

Or Google for similar advice.