I apologize if this is long but I want to detail all the steps that I've done below and hope that someone out there more knowledgeable than me can help spot the problem!
Project: I'm working on a commercial application that I've prototyped completely using an Arduino Uno R3. I drew up a schematic based on available open source Arduino designs that incorporates all the custom components/connectors I'm using in my project to create an Uno-based board using an Atmega8u2-AU and a SMD Atmega328P-AUR.
Board Layout (attached): Please see the two attachments for the board schematics for the Atmega8u2-AU and a SMD Atmega328P-AUR.
Flashing Atmega8u2-AU: The first thing I did was plug in my board to my windows computer using the on board USB port. Windows immediately recognized the board in Device Manager under libusb-win32 devices as ATmega8u2. Next I shorted pins 5 & 6 on my 6 pin J4 header block to reset the 8u2 and heard the characteristic jingle in Windows when the Device Manager refreshed. I then installed the latest version of FLIP and followed the steps to connect to the the 8u2 which it opened the USB connection successfully (see attached screenshot). I then downloaded the Arduino-usbserial-uno.hex file from the Arduino GITHUB page and flashed this to the board. All the steps seemed to have worked perfectly and after disconnecting and connecting the USB cable my computer recognized the board as an Arduino Uno and it installed the proper drivers. In Device Manager, the board is now recognized under Ports (COM & LPT) as Arduino Uno (COM12). For all intents and purposes it looks like my off the shelf Arduino Uno board as far as Windows is concerned.
Flashing Atmega328P-AUR w/Bootloader: I purchased the Olimex AVR-ISP-MKii and used the ICSP10 port and a 6 pin ribbon cable to plug to the J7 header block on my board which corresponds to the 328P. I plugged in my board and opened up Arduino IDE 1.8.9. I have Board set to Arduino/Genuino Uno and Port set to Com12(Arduino/Genuino Uno). If I click Get Board Info I get 0x2341 and 0x0001 for VID and PID. Next I select AVRisp mkII for programmer and plug my mkII to my computer via USB. I followed the Olimex instructions and have the latest drivers installed for the AVR ISP (it shows up in Device Manager as libusb-win32 devices --> AVRISP mkII). Next in the Arduino IDE I clicked Burn Bootloader and everything appears to have flashed correctly in the verbose output terminal with no errors. You can see a log of this report attached. Before when I plugged in my board without the 328P flashed it didnt give me the three LED blink heartbeat but now after flashing the bootloader it does the 3 LED blink when plugged into USB power every time.
Uploading Program: Here is where the trouble starts. I connect to my board as usual without any wires or shields connected (nothing connected to pin 0 or 1 or any other pin except the on-board relay). When I try to upload the blink program I get the following message:
Using Port : COM12
Using Programmer : arduino
Overriding Baud Rate : 115200
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 1 of 10: not in sync: resp=0x32
I see one of the LEDs on the board light up every time it retries this 10 times before it gives up. I have even tried what others have suggested by jumping GND to Reset at various points during upload and still no luck.
Loop Back Test: I performed the Loop Back test found on this forum and it successfully returned what I typed into the serial monitor.
At this point I'm really at a loss for what to try next. I do believe that I have been successful in flashing the 8u2 with the serial/USB communication firmware and that it is working correctly. I also think that the Arduino bootloader was successfully flashed to the 328p. I'm thinking that it might have something to do with my board layout but at this point I have no idea where to start. If anyone can take a look at my attached layouts and offer any insight it would be greatly appreciated!
Using the Arduino IDE you burn the fuses and the bootloader in one operation. The OP isn't using the IDE for the Atmega8u chip so all he did was burn the bootloader.
The README.txt at arduino-1.8.9/hardware/arduino/avr/firmwares/atmegaxxu2/ shows how to write the combined.hex file and also burn the fuses.
Thanks for the help so far everyone. I want to flash the correct fuses to the Atmega8u2 but for some reason I'm not able to connect to the 8u2 using avrdude from the command prompt. Remember the 8u2 is already flashed with the uno serial.hex file but I need to use avrdude to set the fuses correctly. I have done the steps below;
-plug avrispmk2 into 8u2 chip using the ICSP 6 pin ribbon cable (10 pins to 6). The avrispmk2 is recognized in my device manager as libusb-win32 devices > AVRISP mkII
-plug my board into USB to my computer to give the 8u2 power.
-I am running avrdude v5.11 due to some bug issues with the avrispmk2 I was reading about online.
-My avrispmk2 is flashed with AVR456-studio-AVRISP-MKII.hex
I have tested my avrisp by successfully burning the bootloader on the the atmega328p in the arduino IDE.
In command prompt, I enter in: avrdude -p usb82 -c avrispmkii -P usb -v -F -t to enter the terminal to check my current fuses and I get this message belwo:
C:\Users\mike>avrdude -p usb82 -c avrispmkii -P usb -F -v -t
avrdude: Version 5.11-Patch#7610, compiled on Aug 31 2011 at 08:02:19
Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
Copyright (c) 2007-2009 Joerg Wunsch
System wide configuration file is "C:\AVRDUDE\avrdude.conf"
Using Port : usb
Using Programmer : avrispmkii
avrdude: usbdev_open(): Found AVRISP mkII, serno: 000200212345
After about a minute I get this error: avrdude: stk500v2_recv_mk2: error in USB receive
After about 5 minutes I get this error: avrdude: stk500v2_getsync(): timeout communicating with programmer
I know that it is trying to connect to the programmer because if I unplug the avrispmk2 from USB while it is attempting to connect it automatically responds in the command prompt.
A quick update to the previous post. I let the command prompt sit there for 10 minutes and I'm getting the log below...
C:\Users\mike>avrdude -p usb82 -c avrispmkii -P usb -F -v -t
avrdude: Version 5.11-Patch#7610, compiled on Aug 31 2011 at 08:02:19
Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
Copyright (c) 2007-2009 Joerg Wunsch
System wide configuration file is "C:\AVRDUDE\avrdude.conf"
Using Port : usb
Using Programmer : avrispmkii
avrdude: usbdev_open(): Found AVRISP mkII, serno: 000200212345
avrdude: stk500v2_recv_mk2: error in USB receive
avrdude: stk500v2_recv_mk2: error in USB receive
avrdude: stk500v2_recv_mk2: error in USB receive
avrdude: stk500v2_recv_mk2: error in USB receive
avrdude: stk500v2_recv_mk2: error in USB receive
avrdude: stk500v2_recv_mk2: error in USB receive
avrdude: stk500v2_getsync(): timeout communicating with programmer
AVR Part : AT90USB82
Chip Erase delay : 9000 us
PAGEL : PD7
BS2 : PC6
RESET disposition : possible i/o
RETRY pulse : SCK
serial program mode : yes
parallel program mode : yes
Timeout : 200
StabDelay : 100
CmdexeDelay : 25
SyncLoops : 32
ByteDelay : 0
PollIndex : 3
PollValue : 0x53
Memory Detail :
C:\Users\mike>avrdude -p m8 -c avrispmkii -P usb -vv
avrdude: Version 5.11-Patch#7610, compiled on Aug 31 2011 at 08:02:19
Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
Copyright (c) 2007-2009 Joerg Wunsch
System wide configuration file is "C:\AVRDUDE\avrdude.conf"
Using Port : usb
Using Programmer : avrispmkii
avrdude: usbdev_open(): Found AVRISP mkII, serno: 000200212345
avrdude: usbdev_open(): using read endpoint 0x82
I then downloaded the Arduino-usbserial-uno.hex file from the Arduino GITHUB page
Isn’t the current uno usbserial firmware for a 16u2? (If so, it’s strange that it works as much as it does...)
Your symptoms at sketch upload are similar to what happens when auto-reset isn’t working. Do you see the bootloader led flashes when you attempt an upload? Have you tried a manual reset (which has to be Carefully timed...)