ESP8266 Arduino IDE Detection of end of HTTP header

Projects General Guidance
1

Hello, I have a file that resides on my website it is HTTPS enabled.
The file is a simple ascii text file containing text.
I want to read the file via ESP8266 12F module.
I have it connecting ok using wifisecureclient.
But when trying to read the file I get back all the HTTP header text.
I have read that the HTTP header ends with /r/n/r/n
I have used various methods to try and detect this, but all have failed.

while (client.connected()) {
  String line = client.readStringUntil('\n');
  if (line == "\r\n") break;    //if line contains only "\r\n", it's the end of headers
}

Does not work for me. are there other methods. if there are keep it simple like me. Thank you.

2

You should use the genuine HTTPClient, as in this example, which handles all these protocol details.

If you instead prefer to perform HTTP manually, readStringUntil does not include the specified terminator; \n in this case, so the line will never be "\r\n". At most it will be just "\r". But more robust to do this

  String line = client.readStringUntil('\n');
  line.trim();
  if (!line.length()) break;  // blank line means end of headers

Another approach

String payload = client.readString();  // grab everything
int endOfHeader = payload.indexOf("\r\n\r\n");
if (endOfHeader >= 0) {
  payload = payload.substring(endOfHeader + 4);
}

There are several ways this can go wrong though.

3

Thank you for your kind advice, I do like the "There are several ways this can go wrong though." haha trust me this C++ code always seems to go wrong for me. I have managed to find a solution. I found that using a .php script on the server that I call can read the text file off and it comes back after the GET request following the header. But directly before the text file contents there is a {"text":" string which I can check for and directly after the text file text there is a ","timestamp": string.
I used chatgpt to create some code to filter before and after these strings and it came up with working code ! I now have a string variable containing the text.txt contents.