If the brown-out detector is enabled, then the datasheet seems to permit removing the power-stabilisation delay (normally approx 65ms). And with a ceramic resonator the clock-stabilisation delay can be shortened to 1k cycles (normally 16k cycles). So if the power is stable during reset, in theory you'd only have to wait ~63us (at 16MHz).
Usually Arduinos have the BOD set to 2.7V. If running at 16MHz without the power-stabilisation delay it may be wise to increase this to 4.3V.