Trying to program custom atmega320pb board with UNO and/or NANO

right

Where is pin 21 of your 328PB
There are some fake chips going around.
Is the part number atmega328pb-u or atmega328pb-pu ?

As I can see on chip it is atmega328pb-u

Unfortunately it a fake.
There have been other people on the forum with the same problem.

From the 328PB data sheet:

ATmega328PB-AU
ATmega328PB-AUR(3)
ATmega328PB-MU
ATmega328PB-MUR(3)
ATmega328PB-AN
ATmega328PB-ANR(3)
ATmega328PB-MN
ATmega328PB-MNR(3)

So nothing I can do to program it?

Pin 21 should be conneced to ground but I don't see it on your schematic.

Did you forget /RESET (RST)? Pin 10 is not used.

By pin 21 are u refering to pb7? I see that pin has D21 on it in atmega328pb datasheet. If yes that pin is one of the oscillator pins (xtal2) and is connected to the ground via capacitor c8.

D10 of programmer is connected to rst pin of custom pcb board that is being programmed.

When you get time could you send me your sketch please?

I'm sorry, it was difficult to find the right link, because the AVRFreaks forum changed the engine and all the old links stopped working.
However, here is the link where I got the code from:

https://www.avrfreaks.net/s/topic/a5C3l000000UUroEAG/t132446?comment=P-994658

Copy the code from the message, save as arduino sketch and then use with accordance of ArduinoasISP manual.
Just add a wire from pin #3 to XTAL1 on your target AVR.

I have uploaded the code to my NANO I want to use as programmer and connected d3 of NANO to xtal1 of my custom pcb. Still getting an error.

Arduino: 1.8.19 (Windows Store 1.8.57.0) (Windows 10), Board: "Atmel atmega328pb Xplained mini"

C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware\tools\avr/bin/avrdude -CC:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware\tools\avr/etc/avrdude.conf -v -patmega328p -cstk500v1 -PCOM9 -b19200 -e -Ulock:w:0x3F:m -Uefuse:w:0xFC:m -Uhfuse:w:0xDE:m -Ulfuse:w:0xE0:m 



avrdude: Version 6.3-20190619

         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/

         Copyright (c) 2007-2014 Joerg Wunsch



         System wide configuration file is "C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware\tools\avr/etc/avrdude.conf"



         Using Port                    : COM9

         Using Programmer              : stk500v1

         Overriding Baud Rate          : 19200

         AVR Part                      : ATmega328P

         Chip Erase delay              : 9000 us

         PAGEL                         : PD7

         BS2                           : PC2

         RESET disposition             : dedicated

         RETRY pulse                   : SCK

         serial program mode           : yes

         parallel program mode         : yes

         Timeout                       : 200

         StabDelay                     : 100

         CmdexeDelay                   : 25

         SyncLoops                     : 32

         ByteDelay                     : 0

         PollIndex                     : 3

         PollValue                     : 0x53

         Memory Detail                 :



                                  Block Poll               Page                       Polled

           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack

           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------

           eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff

           flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff

           lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00

           hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00

           efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00

           lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00

           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00

           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00



         Programmer Type : STK500

         Description     : Atmel STK500 Version 1.x firmware

         Hardware Version: 2

         Firmware Version: 1.18

         Topcard         : Unknown

         Vtarget         : 0.0 V

         Varef           : 0.0 V

         Oscillator      : Off

         SCK period      : 0.1 us



avrdude: stk500_recv(): programmer is not responding

avrdude: initialization failed, rc=-1

         Double check connections and try again, or use -F to override

         this check.



avrdude: stk500_recv(): programmer is not responding



avrdude done.  Thank you.



Error while burning bootloader.



This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

Could you please check again with standard ArduinoasISP sketch?

I think you changed something in the procedure or in the connections, since there is nothing in the last two sketches that could lead to such an error

I have tried that also, again 0xffffff code. I have set board to atmega328pb xplained mini.

P.S.
I have kept same connections and settings, only thing really changed is the sketch I uploaded from the link you gave me and connected pin d3 to xtal1 of custom pcb.
When I tried with default ArduinoISP code I have disconnected d3 pin.

It is weird that you got

with the both modified sketch. It are the same codes as standard one, but only clock output added.

Sorry, I have no another ideas...

as a last try - can you try to run the modified sketch but not connect pin 3 to the board being flashed?

No luck unfortunately.

Arduino: 1.8.19 (Windows Store 1.8.57.0) (Windows 10), Board: "Atmel atmega328pb Xplained mini"

C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware\tools\avr/bin/avrdude -CC:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware\tools\avr/etc/avrdude.conf -v -patmega328p -cstk500v1 -PCOM9 -b19200 -e -Ulock:w:0x3F:m -Uefuse:w:0xFC:m -Uhfuse:w:0xDE:m -Ulfuse:w:0xE0:m 



avrdude: Version 6.3-20190619

         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/

         Copyright (c) 2007-2014 Joerg Wunsch



         System wide configuration file is "C:\Program Files\WindowsApps\ArduinoLLC.ArduinoIDE_1.8.57.0_x86__mdqgnx93n4wtt\hardware\tools\avr/etc/avrdude.conf"



         Using Port                    : COM9

         Using Programmer              : stk500v1

         Overriding Baud Rate          : 19200

         AVR Part                      : ATmega328P

         Chip Erase delay              : 9000 us

         PAGEL                         : PD7

         BS2                           : PC2

         RESET disposition             : dedicated

         RETRY pulse                   : SCK

         serial program mode           : yes

         parallel program mode         : yes

         Timeout                       : 200

         StabDelay                     : 100

         CmdexeDelay                   : 25

         SyncLoops                     : 32

         ByteDelay                     : 0

         PollIndex                     : 3

         PollValue                     : 0x53

         Memory Detail                 :



                                  Block Poll               Page                       Polled

           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack

           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------

           eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff

           flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff

           lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00

           hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00

           efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00

           lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00

           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00

           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00



         Programmer Type : STK500

         Description     : Atmel STK500 Version 1.x firmware

         Hardware Version: 2

         Firmware Version: 1.18

         Topcard         : Unknown

         Vtarget         : 0.0 V

         Varef           : 0.0 V

         Oscillator      : Off

         SCK period      : 0.1 us



avrdude: stk500_recv(): programmer is not responding

avrdude: initialization failed, rc=-1

         Double check connections and try again, or use -F to override

         this check.



avrdude: stk500_recv(): programmer is not responding



avrdude done.  Thank you.



Error while burning bootloader.



This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

thank you

Pin21 of the ATmega328PB is ground but you schematic symbol does not have a pin 21.

Not sure about pin 21 and ground, I have been using this pinout to check eevrything and gnd pins arent marked with numbers here.

P.S. oh nevermind I see not what you mean, yeah it is connected to GND I have tested it now.