The 8/16U2 is waiting for a certain signal over USB (what is this signal, opening and closing of the serial port?)
That is it.
to pull the main MCUs reset pin low with a virtual DTR pin?. However, how does the host know that the cable has been connected to send out the DTR signal anyway?
The host knows when a USB device is connected. That's an essential part of the whole USB protocol. But enabling DTR is part of the port opening/ closing protocol.
As in, I know how avrdude sends a command to the 16U2 which resets the main MCU, but why does it reset when you open a serial monitor?
EDIT - I think I've just answered my own question whilst typing this - I haven't got a board with me at the moment, but I think I've been stupid... the board doesn't reset when you connect a cable, but when you open a serial monitor in the Arduino IDE?
Indeed - when you open the port, DTR goes active. The download does that and closes it again, then opening the serial monitor does it also.
Why the series cap though? Programmers like the ISP Mk2 don't need that cap in series?
Because they temporarily activate and de-activate the reset according to the ISP protocol precisely as needed, which is not the same protocol as the serial interface to the bootloader.
Is it possible to remove the cap and the diode in the reset circuit, I don't see why it needs to be there? If a chip like an FTDI230X can just pull the line low, surely a reset occurs?
You have an attitude problem there. If the system is designed in a certain way, there is a critical reason for it to be designed that way, so it is very unwise to imagine "improvements" before you have thought it all out.
In this case the reason is perfectly straightforward - you do want to reset the chip, but the chip will not operate whilst reset - you have to allow the chip to "un-reset" before it will execute code, including running the bootloader.