ATmega328P, Optiboot, stk500_recv() and going deep

Hold the phone.

A data overrun occurs when the receive buffer is full (two characters), it is a new character waiting in the receive shift register, and a new start bit is detected.

This butchered sentence seems to reveal an important idiosyncrasy. My interpretation is the same as this one, though it was never confirmed.

That is, there is actually 3 bytes stored in the UART when an overrun occurs: the 2 in the FIFO and another one in the shift register! If so, then the three sync commands will fill the UART like so:

F0|F1|SR  : FIFO 0 (the next to be read), FIFO 1, shift register
--|--|--
30|20|XX  : Both bytes of 1st sync command arrives. No overrun.
30|20|30  : 1st byte (0x30) of 2nd sync command arrives. FIFO is full.
30|20|20  : 2nd byte (0x20) of 2nd sync command arrives. Previous byte is lost.
30|20|30  : 1st byte (0x30) of 3rd sync command arrives. Previous byte is lost.
30|20|20  : 2nd byte (0x20) of 3rd sync command arrives. Previous byte is lost.
20|20|XX  : 1st byte (0x30) is read. Shift register moved into FIFO.
20|XX|XX  : 2nd byte (0x20) is read. First sync command correctly parsed.
XX|XX|XX  : 3rd byte (0x20) is read. Uh oh. Due to overrun, this
          : is actually the 2nd byte of the 3rd sync command.

This seems to be the behaviour observed here - if six bytes are sent, and then all are read, you get the 1st, the 2nd, and then the last!

So that, if true, would explain it. And maybe it would explain why for decades there have been reports on this forum of mysterious "stk500_recv(): programmer is not responding" issues relating to baud rate / cpu frequency config. If the LED pulses overlap with avrdude's sync commands (and they very nearly do even in the usual case) you'll get incomplete comms.