vulture2600:
Thank you for the replies. All of you seemed to answer each other's questions. I had everything in F() macros at one point but I removed them because I've read its quite slow compared to not using it. I first removed it from XML_response() because I wanted the web server to respond quickly.
I note that you are not making much of an attempt to parse the incoming HTTP request. It's possible that your server is getting hit by something unexpected - a web spider or something, or maybe your phone sends out the occasional HTTP OPTIONS request. If anything sends a POST to your web server, your code will slurp the request line and HTTP headers, respond, and then resume listening to the connection. But then the remaining content will still be coming down the line. If that content doesn't have a blank line in it, then the code will sit there waiting for the 'next' request to complete.
I would also be inclined to add a timeout
startOfRequestMs = millis();
while (cl.connected() && millis() - startOfRequestMs < /* say */ 5000) {
}
five seconds is more than enough time for anyone to send a simple GET request.
Hmm ...
I see you are using Keep-Alive. Do you really need it? I mean, it's handy if something is going to be making a stream of separate requests one after another. But is that what this is going to be doing? If you don't need it, you can just close the connectikon every time after you have sent your response.
In any case: I suggest adding some timeouts to close the connection impolitely if the thing talking to your server seems to be acting screwy. It would also be a good idea to manage HTTP properly and read the Content-Length header.
The kind of behaviour you are seeing could very well be caused by network dropouts that your code does not recover from.
-- EDIT --
Dude, you cannot use Keep-Alive if you are not sending a Content-Length header. The client on the other end has no way to know when the page has finished other than by counting the incoming bytes after the HTTP headers or by waiting for the connection to close. I would guess that the web browser on your phone handles this by timing out. The giveaway that this is happening is if the page seems very slow to load.
If you are not closing the connection after sending the response, then you must send a Content-Length HTTP header.
-- EDIT 2 --
Also, I note that you are sending a CORS header. Are you familiar with the CORS pre-flight check? Your code does not seem to have anything there to intercept the OPTIONS request that is part of that sequence. I don't know if that's relevant … but it's a thing. Because the check is an OPTIONS request, you mustn't send any body content to that request - only the headers. If you are unconditionally sending the XML content, well, bad things can happen. Bytes left lying around in the pipe unaccounted for.