Okay, updates.
Trying to solve this without broadcasting a Ready signal as @PaulS suggested (it's a good workaround suggestion, I just want to understand what's happening here).
Setting comPort.DtrEnable to true seems to fix the random ? characters that were prepended to the serial write.
However, when the Arduino is pinged after the port is opened, nothing seems to come back. The serial write never appears to get to the Arduino—possibly because it's just taking too long for the port to open.
This is backed up by the fact that if I set a breakpoint and step through any function that opens the serial port, the DataReceived handler will fire. So it seems to be a question of some required delay.