Certainly, C# (or other application that maintains state) is able to maintain an open connection. I was concerned that each invocation of the script opened and closed the serial port.
Can you see that the Arduino does indeed restart each time the script is invoked? Watch the onboard LEDs that flash when the Arduino starts. If they flash the same way when the script is invoked, then the script is opening and closing the port each time.