Slow response by arduino server being controlled by python client

[Grumpy_Mike]

Is it possible that the serial printouts are actually limiting it? Or does it have to do with how I'm opening/closing/listening to the port

Yes both things will slow it down.