400 Bad Request after web provider performed server maintenance.

I’m working on a project where an Arduino-compatible device (NodeMCU ESP8266) connects to a web server over SSL and retrieves data from a php file. The data returned is simply a number or series of numbers. The values are then displayed on a series of 7-segment displays. Everything was working until around 2am on Thursday, September 19th. After that time, instead of getting data back from the php script, I’m getting a 400 Bad Request. I put a couple entries in my php script that should write to error_log and I’m getting nothing on the web server side, so I’m pretty confident I’m not even reaching the file.

I don’t know what application versions were in place prior to the updates but I know that these are the current versions:

Apache Version 2.4.39
PHP Version 7.3.8

request:
GET /assets/scripts/php/checkTasks.php?action=getRows&uid=esp8266 HTTP/1.0
Host: battletech-live.net
Connection: close

For HTTP, I’ve tried 1.0, 1.1 and 2.0 but they don’t seem to make a difference. Prior to the web hosting provider updates, it was working fine with SSL and I was working with setInsecure so that I could retrieve data even with an invalid fingerprint. Both were working. The best I can figure is that something needs to change in my request, but I don’t know what.

response:
HTTP/1.1 400 Bad Request
Date: Fri, 20 Sep 2019 23:47:41 GMT
Server: Apache
Content-Length: 422
Connection: close
Content-Type: text/html; charset=iso-8859-1

400 Bad Request

Bad Request

Your browser sent a request that this server could not understand.

Additionally, a 400 Bad Request error was encountered while trying to use an ErrorDocument to handle the request.


Apache Server at host2011.hostmonster.com Port 443

Bits of code:
WiFiClientSecure client;
client.setFingerprint(fingerprint);
// client.setInsecure();

if (client.connect(host, httpsPort)) {

String request = String("GET “) + url + " HTTP/1.0 \r\n” + "Host: " + host + “\r\n” + “Connection: close\r\n\r\n”;
client.print(request);
Serial.print(request);
while (client.connected()) {
// String line = client.readStringUntil(’\n’);
String line = client.readString();
Serial.println(line);
}
}else{
Serial.println(“connected failed”);
}

change HTTP/1.0 to HTTP/1.1

This morning, I configured one of my local CentOS boxes for SSL and stripped my sketch down to the most basic components. I wanted to rule out my hosting provider as being the source of the problem. As far as an HTTP declaration is concerned, I’ve tried 1.0, 1.1 and 2.0.

Below is my code, and the serial output of the sketch.

#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>

const char* ssid = “--------”;
const char* password = “--------”;
const char* host = “192.168.2.201”;
const int httpsPort = 443;
String url = “/battletech-live.fingerprint”;

// SHA1 fingerprint of the certificate
char fingerprint PROGMEM = “8F F5 C4 20 BA 96 D3 D7 F6 C8 DA 3B FE 1E F8 4B 77 08 CD 5F”;

void setup() {
Serial.begin(115200);
WiFi.mode(WIFI_STA);
WiFi.setOutputPower(10);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println(“WiFi connected”);
Serial.print("IP address: ");
Serial.println(WiFi.localIP());

// Use WiFiClientSecure class to create TLS connection
WiFiClientSecure client;
client.setFingerprint(fingerprint);
// client.setInsecure();

if (client.connect(host, httpsPort)) {
Serial.print("connected to ");
Serial.println(host);
String request = String("GET “) + url +
" HTTP/1.1 \r\n” + "Host: " +
host + “\r\n” + “Connection: close\r\n\r\n”;
client.print(request);
while (client.connected()) {
String line = client.readString();
Serial.println(line);
}
} else {
Serial.println(“connected failed”);
}
}

void loop() {

}


WiFi connected
IP address: 192.168.4.124
connected to 192.168.2.201

HTTP/1.1 400 Bad Request
Date: Sat, 21 Sep 2019 16:03:58 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/5.4.16
Strict-Transport-Security: max-age=63072000; includeSubdomains
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Content-Length: 226
Connection: close
Content-Type: text/html; charset=iso-8859-1

400 Bad Request

Bad Request

Your browser sent a request that this server could not understand.


The output I get from my hosting provider is a little different.

WiFi connected
IP address: 192.168.4.124
connected to battletech-live.net

HTTP/1.1 400 Bad Request
Date: Sat, 21 Sep 2019 16:09:36 GMT
Server: Apache
Content-Length: 422
Connection: close
Content-Type: text/html; charset=iso-8859-1

400 Bad Request

Bad Request

Your browser sent a request that this server could not understand.

Additionally, a 400 Bad Request error was encountered while trying to use an ErrorDocument to handle the request.


Apache Server at host2011.hostmonster.com Port 443

and the space after HTTP/1.1?

A valid HTTP 1.1 request MUST at least include this:

GET /path/document.htm?some=query HTTP/1.1
host: www.thehost.com

EDIT: I see no call to WiFiClientSecure.setCACert() which apparently is mandatory in order to use TLS.

EDIT2: An example.

I'm making some progress. I rewrote the request and it's now pulling the correct value.

String request = String("GET ") + url + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Connection: close\r\n\r\n";
client.print(request);

However, what I'm getting back is more than I want. All I want is the number at the end, not the header information.

HTTP/1.1 200 OK

Date: Sat, 21 Sep 2019 17:10:40 GMT

Server: Apache

Upgrade: h2,h2c

Connection: Upgrade, close

Content-Length: 1

Content-Type: text/html; charset=UTF-8

2

void getPendingTasks() {
WiFiClientSecure client;
client.setFingerprint(fingerprint);
if (client.connect(host, httpsPort)) {
String request = String("GET ") + url + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Connection: close\r\n\r\n";
client.print(request);
while (client.connected()) {
String line = client.readStringUntil('\n');
int ct = line.toInt();
Serial.println(ct);
}
}
}

Results:
0
0
0
0
0
0
0
0
2

so you removed the space

Yes, it looks like the space might have been the main cause.

If you only want the value of the last line, then should be able to do this:

String line;
while (client.connected()) line = client.readStringUntil('\n');
int value = line.toInt();
Serial.print("Value is: ");
Serial.println(value);

Thank you for all the help. Danois90, your solution is really close to what I came up with too. I didn’t put it all on one line, but it works.

String request = String("GET “) + url + " HTTP/1.1\r\n” +
"Host: " + host + “\r\n” +
“Connection: close\r\n\r\n”;
client.print(request);
while (client.connected()) {
line = client.readStringUntil(’\n’);
}
int ct = line.toInt();

From there, I’m able to send the value to a writeDigit function to display the value on 7-segment displays.