I need some help to "GET"

Using Arduino Networking, Protocols, and Devices
1

I am not knowledgeable in HTTP protocol programming, hope someone can offer some expertise here. My ESP32 project’s been successfully using GET to download text from the website aviationweather.gov… up to a few days ago. The site changed the protocol and now I am unable to figure out how to download data from it. The site's API specifies:
Curl
curl -X 'GET'
'https://aviationweather.gov/api/data/metar?ids=KSJC'
-H 'accept: text/plain'
Request URL
https://aviationweather.gov/api/data/metar?ids=KSJC

In my sketch, I am sending the request:
GET https://www.aviationweather.gov/api/data/metar?ids=KSFO HTTP/1.1

In return, I am getting a 400 error and the following:

Header:
HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=us-ascii
Date: Fri, 20 Oct 2023 04:21:15 GMT
Content-Length: 334
Body:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid Hostname</h2>
<hr><p>HTTP Error 400. The request hostname is invalid.</p>
</BODY></HTML>

The API specs and examples can be found here:
https://aviationweather.gov/data/api/#/Data/dataMetars

What do I need to send in my “GET” request. Any assistance will be much appreciated.

2

how do you send it? show us the sketch

3

After a successful SendCertCertificate(), then I have:

        // Make the HTTPS request
        String sGetReq = "GET https://";
        sGetReq += "www.aviationweather.gov";
        sGetReq += "/api/data/metar?ids=";
        sGetReq += "KJFK";
        sGetReq += " HTTP/1.1";
        client.println(sGetReq);

        client.println("Connection: close\n");

        while (client.connected())     // Consume the header (end at blank line)
        {
            sLine = client.readStringUntil('\n');
            if (sLine == "\r")
            {
                break;
            }
        }

        // Keep fetching lines as available, parse each for the key sICAO
        while (client.available())
        {
            sLine = client.readStringUntil('\n');

            // Parse the text, etc...
        }
4

Please get a HTTPS client library and stop doing the request by hand.

Anyway does this work better?

GET /api/data/metar?ids=KSJC HTTP/1.0
host: aviationweather.gov
accept: text/plain
5

use the HTTPClient library bundled with the esp32 core

6

Thank you both for the suggestion. This works:

WiFiClientSecure *client = new WiFiClientSecure;

if (client)
{
    client->setCACert(sROOT_CERT);
    HTTPClient https;

    if (https.begin(*client, "https://aviationweather.gov/api/data/metar?ids=KJFK"))
    {
        int16_t responseCode = https.GET();
        if (responseCode > 0)
        {
            if (responseCode == HTTP_CODE_OK || responseCode == HTTP_CODE_MOVED_PERMANENTLY)
            {
                // Get payload from server
                sLine = https.getString();
            }
        }
    }
 
    https.end();
}
delete client;
7

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.