PaulS:
What confuses me is why available() seems to never return true after the data has started to stall, but presumably at some point we'd expect it to become true when more data was available in the buffer from the source?
First, available() returns the number of bytes available to read, not true or false.
Second, this really looks like a failure on the part of the server to send all the data.
Apologies, you're right (although technically I think we can both agree that available() returns something that can be considered 'truthy').
However, I'm pretty sure that it's not the server stalling. I'll see if I can produce some Wireshark logs to demonstrate this, but anecdotally for the moment, what I see is the server and arduino sending packets and ACKing, but then the arduino fails to ACK, and the server retransmits the latest packet again and again until the connection is closed.
I'll try to get a concrete TCP log to hopefully demonstrate this.