is *A-PU supported by optiboot?
"Probably."If you're asking about the "A" versions (atmega168-PU vs atmega168a-PU), those are normally indistinguishable (same behavior, same signature, etc - the bootloader and Arduino core SW can't even tell.) (Alas, not true of many "B" versions ("Atmel, you fool!"))If you're asking about the part after the dash, that usually indicates package type, and the software doesn't care...
Which dev board?
This is a blog on programming AVRs on breadboards that includes internal clock how-to,https://gammon.com.au/breadboardSee if your errors show up on that page.
You can program your custom ATmega328pb board using minicore together with an Arduino uno with the ISP sketch so you can, for example, load the blink sketch?So it is not easy to see why you fail to be able to load optiboot.The only thing I could suspect is that the resistor to the led on the SCK pin (pb5) is rather strong at 220 ohms. 1k is usual on say a Nano. This may interfere with attempts to program it. That your pdf attachment shows inconsistent device signatures could support that idea.
Do the custom boards have socketed AVR's like the Uno?
. . .As i am a newbie, i tried to look through the datasheet as to figure out how did you come to that conclusion, but cannot really understand it. I think it will take a lot more experience on my end to reach that stage
This sketch was inspired by the Optiloader sketch written for the Arduino. However it is a total rewrite, in order to accomodate the Mega2560 board, which the original one did not handle, due to the larger address space.Supported bootloadersThe code for the following bootloaders is incorporated in the sketch, and will be downloaded depending on which signature is detected:Atmega8 (1024 bytes)Atmega168 Optiboot (512 bytes)Atmega328 Optiboot (for Uno etc. at 16 MHz) (512 bytes)Atmega328 (8 MHz) for Lilypad etc. (2048 bytes)Atmega32U4 for Leonardo (4096 bytes)Atmega1280 Optiboot (1024 bytes)Atmega1284 Optiboot (1024 bytes)Atmega2560 with fixes for watchdog timer problem (8192 bytes)Atmega16U2 - the bootloader on the USB interface chip of the UnoAtmega256RFR2 - the bootloader on the Pinoccio Scout boardYou have the option of programming (writing) the bootloader or verifying (checking) the existing bootloader.
It is not in the data sheet. The most concrete information about shared use of SPI lines, that I can find, is here: http://ww1.microchip.com/downloads/en/appnotes/atmel-2521-avr-hardware-design-considerations_applicationnote_avr042.pdf , although this may not exactly cover your scenario.However, there is a also lot of anecdotal evidence about the wisdom of keeping clear of the SPI lines when possible to minimise the risk of interference with attempts to program the device, for example: https://www.avrfreaks.net/forum/can-i-reuse-isp-pins-other-taskstarget-atmega164p?skey=reuse%20of%20spi%20linesAs a result of my own experience, I attempt to keep at least 1K away from the SPI lines where possible. I documented some of this here: https://forum.arduino.cc/index.php?topic=673817.0 (post#6 onwards)