I am building a real-time rail departure board. The project is based on an project by GitHub - nichelaboratory/GxEPD2_Train_Departures_Board: Display UK train departures using ESP32 + ePaper Display.
I have modified it to use a ESP32 Smart Display or "Cheap Yellow Display" (CYD). The real-time information is provided by www.transportapi.com in JSON format. A typical HTTP request required by the API is of the form:
curl -X GET "https://transportapi.com/v3/uk/train/station_timetables/crs%3ABTH.json?from_offset=PT00%3A30%3A00&to_offset=PT01%3A30%3A00&limit=10&live=true&train_status=passenger&station_detail=destination&type=departure&app_key=f50c4bebb4893a83aa89ef5d5b76d2ee&app_id=af080d07"
-H 'accept: application/json' .
When I use this URL using the Windows Command Prompt Curl - JSON data is returned successfully.
However I am unable to enter this HTTP request using WiFiSecure - client.println(GET....). I get a connection to the server but an error 400 is returned indicating that the request is not understood.
How can I the structure the HTTP request to work with WiFi secure?
It is impossible to suggest anything without seeing your code
The fact that you're getting 400 means that the secure connection to the HTTP server is working, which is what trips up a lot of people.
The server will usually tell you what it doesn't like about your request. Follow some examples to read the body of the response. If you need more help, Include it along with your sketch, posted as <CODE/>
I have included my code - which is the ESP32 WiFiClientSecure example. A bit more background to this:
The API I'm trying to obtain live rail departure data provides a tool to produce the necessary HTTPS request to download in JSON format for the required data. The example I have chosen is:
curl -X GET "https://transportapi.com/v3/uk/train/station_timetables/crs%3ABTH.json?from_offset=PT00%3A30%3A00&to_offset=PT01%3A30%3A00&limit=10&live=true&train_status=passenger&station_detail=destination&type=departure&app_key=f50c4bebb4893a83aa89ef5d5b76d2ee&app_id=af080d07"
-H 'accept: application/json'
When this is sent using Command prompt >curl the data is returned correctly.
I have transposed this into the WiFiClientSecure client.print("Get.....) as best I can and this is probably why it's going wrong! Using the attached program the following is returned:
Starting connection to server...
Connected to server!
headers received
400 - Bad Request | Your browser sent a request this server could not understand.
�
#include <WiFiClientSecure.h>
#include <esp_crt_bundle.h>
#include <ssl_client.h>
const char* ssid = "xxxxxxxxxxx"; // your network SSID (name of wifi network)
const char* password = "xxxxxxxxxx"; // your network password
const char* server = "www.transportapi.com"; // Server URL
// www.howsmyssl.com root certificate authority, to verify the server
// change it to your server root CA
// SHA1 fingerprint is broken now!
const char* test_root_ca= \
"-----BEGIN CERTIFICATE-----\n" \
// Not included to save space
"-----END CERTIFICATE-----";
WiFiClientSecure client;
void setup() {
//Initialize serial and wait for port to open:
Serial.begin(115200);
delay(100);
Serial.print("Attempting to connect to SSID: ");
Serial.println(ssid);
WiFi.begin(ssid, password);
// attempt to connect to Wifi network:
while (WiFi.status() != WL_CONNECTED) {
Serial.print(".");
// wait 1 second for re-trying
delay(1000);
}
Serial.print("Connected to ");
Serial.println(ssid);
client.setCACert(test_root_ca);
//client.setCertificate(test_client_cert); // for client verification
//client.setPrivateKey(test_client_key); // for client verification
Serial.println("\nStarting connection to server...");
if (!client.connect(server, 443))
Serial.println("Connection failed!");
else {
Serial.println("Connected to server!");
// Make a HTTP request:
client.print ( " -X GET https://transportapi.com/v3/uk/train/station_timetables/crs%3ABTH.json?from_offset=PT00%3A30%3A00&to_offset=PT01%3A30%3A00&limit=20&live=true&train_status=passenger&station_detail=destination&type=arrival&app_key=f50c4bebb4893a83aa89ef5d5b76d2ee&app_id=af080d0");
client.println(" -H 'accept: application/json'");
client.print("Host: ");
client.println(server);
client.println("Connection: close");
client.println();
while (client.connected()) {
String line = client.readStringUntil('\n');
if (line == "\r") {
Serial.println("headers received");
Serial.println(line);
break;
}
}
// if there are incoming bytes available
// from the server, read them and print them:
while (client.available()) {
char c = client.read();
Serial.write(c);
}
client.stop();
}
}
void loop() {
// do nothing
}
-X and -H are command-line switches specific to curl. They are not part of HTTP. (And it defaults to using GET, so you don't need the -X.)
Since you are using WiFiClient::println, you are manually performing HTTP. To see what actually goes "over the wire", add curl's -v switch to enable verbose output. And add --http1.1 in case the server supports HTTP/2 (it does in this case). curl will use it, but it's actually not what you're doing.
$ curl -v --http1.1 'url-in-quotes'
* Trying 13.226.225.38:443...
* Connected to transportapi.com (13.226.225.38) port 443 (#0)
...
... a bunch of stuff
...
* using HTTP/1.1
> GET /v3/uk/train/...trimmed.a.bunch... HTTP/1.1
> Host: transportapi.com
> User-Agent: curl/7.88.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.1 200 OK
< Content-Type: application/json
> is request, < the response, and * are "comments".
- the first line
- starts with the verb
- has the path starting with a slash and the query parameters
- no need for the protocol or host since you are already connected
- ends with the specific
HTTPversion
- then the headers, without the
-HHostis required for HTTP 1.1- It should include
Connection: closeas in the example, since that is what you're actually doing with theclient.stop(). Normally a browser using 1.1 doeskeep-aliveinstead to reuse the connection to request the next thing.
- ending with a blank line
Also note the response defaults to JSON, so the Accept header is not required, but shouldn't hurt.
Starting with GET is right; the rest is wrong. Re-read the part of my post above describing "the first line"; refer to the verbose output from curl, the line starting with > GET
Thanks for you help. I got it to work
This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.