Receiving HTTP GET on ESP32 from a Server

Hey all,
I have an auto irrigation project using ESP32, I'm trying to send a command from a server to the ESP32 to open or close a certain pin, I have used this code and it works when I'm connected to my internal network, but when trying to access the ESP32 when not connected to the local WiFi I'm unable, even when opening the relevant ports in the router and firewall, anyone has any idea how to set up the ESP32 to be able to receive command from a non-local server?

Thanks.

/*
WiFi Web Server LED Blink

A simple web server that lets you blink an LED via the web.
This sketch will print the IP address of your WiFi Shield (once connected)
to the Serial monitor. From there, you can open that address in a web browser
to turn on and off the LED on pin 5.

If the IP address of your shield is yourAddress:
http://yourAddress/H turns the LED on
http://yourAddress/L turns it off

This example is written for a network using WPA encryption. For
WEP or WPA, change the Wifi.begin() call accordingly.

Circuit:
by Tom Igoe

ported for sparkfun esp32
31.01.2017 by Jan Hendrik Berlin

  • WiFi shield attached
  • LED attached to pin 5

created for arduino 25 Nov 2012

*/

#include <WiFi.h>

const char* ssid = "";
const char* password = "";
int Sensor = 4;
WiFiServer server(9999);

IPAddress local_IP(X, XX, XXXX, XXXX);

// Set your Gateway IP address
IPAddress gateway(X, XX, XXX, XXXX);

IPAddress subnet(255, 255, 255, 0);

void setup()
{
Serial.begin(115200);
pinMode(Sensor, OUTPUT); // set the LED pin mode

delay(10);

// We start by connecting to a WiFi network

// Configures static IP address
if (!WiFi.config(local_IP, gateway, subnet)) {
Serial.println("STA Failed to configure");
}

// Connect to Wi-Fi network with SSID and password
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
// Print local IP address and start web server
Serial.println("");
Serial.println("WiFi connected.");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
server.begin();
}

int value = 0;

void loop(){
WiFiClient client = server.available(); // listen for incoming clients

if (client) { // if you get a client,
Serial.println("New Client."); // print a message out the serial port
String currentLine = ""; // make a String to hold incoming data from the client
while (client.connected()) { // loop while the client's connected
if (client.available()) { // if there's bytes to read from the client,
char c = client.read(); // read a byte, then
Serial.write(c); // print it out the serial monitor
if (c == '\n') { // if the byte is a newline character

      // if the current line is blank, you got two newline characters in a row.
      // that's the end of the client HTTP request, so send a response:
      if (currentLine.length() == 0) {
        // HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
        // and a content-type so the client knows what's coming, then a blank line:
        client.println("HTTP/1.1 200 OK");
        client.println("Content-type:text/html");
        client.println();

        // the content of the HTTP response follows the header:
        client.print("Click <a href=\"/H\">here</a> to turn the LED on pin 5 on.");
        client.print("Click <a href=\"/L\">here</a> to turn the LED on pin 5 off.");

        // The HTTP response ends with another blank line:
        client.println();
        // break out of the while loop:
        break;
      } else {    // if you got a newline, then clear currentLine:
        currentLine = "";
      }
    } else if (c != '\r') {  // if you got anything else but a carriage return character,
      currentLine += c;      // add it to the end of the currentLine
    }

    // Check to see if the client request was "GET /H" or "GET /L":
    if (currentLine.endsWith("GET /H")) {
      digitalWrite(Sensor, HIGH);               // GET /H turns the LED on
    }
    if (currentLine.endsWith("GET /L")) {
      digitalWrite(Sensor, LOW);                // GET /L turns the LED off
    }
  }
}
// close the connection:
client.stop();
Serial.println("Client Disconnected.");

}
}

There should be no need to change anything on the ESP. It will be router configuration. When I did something similar, I had to specify that incoming port 80 traffic was to be directed to the static address of my web server.

Thanks for the quick replay, I don't see such an option on my router, I defined the ESP as static and reserved this IP as static on my router, as seen here:

And also opened the 9999 port as open to the specific IP from which I'm supposed to get the GET from, but still it doesn't work, however when I set up a Web Server on my PC and do the same process through the router I am able to send HTTP GET to the web server on my PC, so I assumed it must be something in the ESP32

That's odd. By the time any packet traffic reaches the ESP, it should be indistinguishable from anything local. The router is responsible for directing any response back to the wan.

Do you see anything on serial about a new client?

nope, no response at all when not connected through the local network.

I would be inclined to get a copy of wireshark running on a wifi connected machine on your LAN and see where the packets are going.

It might be worth running a web server on such a machine first to verify that your router will forward wan traffic over wifi.

The second part I already tried with a web server running on my PC and the router managed to pass the communication ok, the first part about wireshark I'm not sure how to do, any quick guide on how to use wireshark?

Is your PC WiFi or hard wired ethernet? It may make a difference.

As to Wireshark, I'm no expert - I usually install it, set it to promiscuous mode and start capturing packets. The filters are probably worth a look, but scrolling through what's there can work too as long as there isn't much going on on your network.

PC is hard wired ethernet but it also worked from my iphone when connected to local WiFi, will get WireShark analysis soon.

I recall (vaguely) doing some Wireshark testing and being surprised that I didn't see wireless traffic on the wired network. I had to put the sniffer on a Pi to view it.

Given that, I don't think your testing so far is conclusive, I would want to see a wifi connected web server responding to wan requests.

I can only do the test on Monday, however after playing a bit more with it I now get an active refusal from the server, does that help you in any way to pinpoint the problem?

To be clear, this is with port forwarded as I shared before, and Mcafee + windows firewall turned off completely.

I had a similar issue recently and found that to use anything other than port 80 I had to use the header

client.write("HTTP/1.1 200 OK\r\n");
client.write("Content-Type: text/html\r\n");
client.write("Connection: close\r\n");
client.write("\r\n");
client.write("<!DOCTYPE HTML>\n");

where exactly in the code would you insert these lines?

Ok so I managed to get it to work using this guide, which uses ngrok:

However, I still am unable to get it to work without ngrok, any ideas?

I haven't used ngrok, but a brief look suggests that it's living inside your firewall and setting up a connection from your network and forwards packets to your ESP. Blynk does something similar. It's nice because you don't need to poke a hole in your firewall.

That suggests to me that your issue is still router configuration. I'd get Wireshark working and look at traffic when you have a hard wired web server going and compare that with what happens if the web server is wireless.