ESP8266 custom web server

Are there any good/modern guides that explain how to send/get data from a custom website like test.com and where the front-end will be hosted on the website instead of the writing the html inside the .ino file...

A simple toggle ON/OFF for a led would do...

Everything that I find is either completely outdated or hosted on the computer IP (html inside the arduino file)..

Do yourself a favor, and read up on the ESP8266AsyncWebServer. It is about 1000X more useful than the very simple ones, and allows you to define the site with actual files (HTML, CSS, javascript, etc.), rather than having to hard-code the HTML into the application. The Asyncserver is MUCH easier to setup and use, and orders of magnitude more flexible and powerful.

thank you for your reply.

Yet another library to read on, it is extremely overwhelming for new comers to find their way into all of these libraries, it seems for every comma there is a new library. Also the documentations are not user-friendly at all. Basically a readme file on github pages and you find your way through..

The documentations are an absolute nightmare. That's why I'm seeking for visual tutorials..

There are plenty of tutorials and example applications for ESP8266AsyncWebServer. Google will reveal all...

@RayLivingston I did, however every single one I saw so far is hosted locally.. Nothing from an actual web server. Hell, it might not even be possible.

That makes no sense... If you are running an 8266 web server, it can ONLY be hosted locally. That is the whole point. If you're hosted on an "actual web server" what do you need an 8266 web server for??

You need to better explain EXACLTY what it is you're trying to accomplish.

@RayLivingston Basically send request's from a website (not hosted locally) such as toggling a led on/off...

Here is a video I found that does exactly that I think, however some stuff is deprecated...

In that scenario, the ESP8266 is the client, NOT the server. And it would have to parse, and understand, the HTML being sent by the website. You would have to design the HTML interface on the server side to do whatever it is you want to do, and make it simple enough to be parsed by the 8266. In effect, the 8266 acts as a browser, to request a web page, parse it, send GET/PUT requests to it, and parse the responses.

@RayLivingston I never said the ESP8266 is the server? I already have my personal website and I understand that I need to send the data and then read it on the ESP8266. My problem is I don't know how to send data from my website to the ESP8266.... All the guides are basically showing the process of sending requests where both the server (website) and ESP8266 are hosted locally...

That is a basic HTTP question. The website responds to HTTP PUT and GET requests. Some requests return web pages, some simply return data. What each request does is TOTALLY up to the web site. The client (the 8266) has to know what requests the server understands. It sends the request, waits for the response, and parses it. The ESPAsyncTCP library can be used to send properly formatted requests, and it will handle the parsing of simple responses, and invoke user-provided callback functions to act on the responses. The ClientServer example in the ESPAsyncTCP library sketches gives you the basics, that will let you send requests and handle responses without having to understand the underlying protocols and data formats. You would need only the client side sketch to do what you want, since you want to use a "real server" for the website.

Therefore this is not an Arduino question at all

I'm not sure what you really want to achieve.

Do you want to serve the page on another webserver and that server should call the esp to switch pins?

so
[browser] --> [3rd party webserver with HTML page] --> [your esp in your LAN]

[browser] --> [3rd party webserver with HTML page] --> [your esp in your LAN]

yes exactly that and exactly the video I sent above as well where I can toggle pins on the ESP from my webserver (not hosted locally).

After looking for a day, what I managed to do so far is in a loop make the ESP visit a page of my website example.com/data every 5 seconds and from there read the data and react accordingly.

But, I don't think visiting a page every 5 seconds to get the data is the most efficient way, isn't there a similar option for event listeners on the ESP? Or maybe I should look into websockets instead?

You might be able to use server-side PHP code to talk to your ESP module at a certain TCP/UDP port

PHP Sockets

a)

this is a little different to

b)

because this looks more like

[browser] --> [3rd party webserver with HTML page] --> [persist data on 3rd party data storage]
async:
[ESP8266httpclient] -->[3rdparty webserver resource] --> [retrieve data from 3rd party data storage]

advantages: you need no port forwarding on your router, no (dyn)DN
disadvantage: it's asyncron

So you should decide what you really want.

do you still want to go way A?

@noiasca I'm totally confused now..

but yeah I think I want to A route..

ok then a plan:

a) I would start wirth a plain lokal ESP8266webserver Example, because it will contain 90% of what you will need on the ESP8266. you can test it locally. A page with a Form or simple HREF's to a resource on your ESP server to get the commands (your button presses).

b) if that works, make your ESP accesable from internet (router port forwarding, dyndns or fixed IP)

c) if this works

  • you can simple publish a page on your external webserver with a form with buttons, and posting (or a simple GET Request) to your ESP8266. Just call the same page on the ESP8266 like you would call in the form on the ESP.

As support for a)
first, see the example ESP8266WebServer / HelloServer - make it working.
If you want to see how I do a webserver on the ESP, see
https://werner.rothschopf.net/microcontroller/202108_esp_generic_webserver.htm
try

#define USE_BOARD 99

for the beginning. This variant should work right of the box.

Tell us when you have your esp webserver up and running.

@noiasca thank you for your help.

this is the part that I don't know how to accomplish. How to send a GET/POST request's from my website to the ESP8266 and catch them.

That's why I tried this alternative: I store the pin numbers in my database and simply update them to HIGH or LOW with forms on the website itself. And then visiting my website page example.com/data from the ESP8266 every 5 seconds and on the example.com/data page, I was displaying data from my database as json.

Here is my current code:

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClient.h>
#include <Arduino_JSON.h>

#include <ESP8266WiFiMulti.h>
ESP8266WiFiMulti WiFiMulti;

 const char* SSID = "SSID";
 const char* PASSWORD = "PASSWORD";

// Your IP address or domain name with URL path
const char* serverName = "http://example.com/data";

// Update interval time set to 5 seconds
const long interval = 5000;
unsigned long previousMillis = 0;

const int ledPin = 5;

String outputsState;


void setup() {
  Serial.begin(115200);

  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, HIGH);
  
  WiFi.mode(WIFI_STA);
  Serial.print("Connecting to ");
  Serial.println(SSID);
  WiFi.begin(SSID, PASSWORD);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("Connected to WiFi");
}

String httpgrequest(const char* serverName) {
  WiFiClient client;
  HTTPClient http;
    
  // Your IP address with path or Domain name with URL path 
  http.begin(client, serverName);
  
  // Send HTTP POST request
  int httpResponseCode = http.GET();
  
  String payload = "{}"; 
  
  if (httpResponseCode>0) {
    Serial.print("HTTP Response code: ");
    Serial.println(httpResponseCode);
    payload = http.getString();
  }
  else {
    Serial.print("Error code: ");
    Serial.println(httpResponseCode);
  }
  // Free resources
  http.end();

  return payload;
}

void loop() {
  unsigned long currentMillis = millis();
  
  if(currentMillis - previousMillis >= interval) {
     // Check WiFi connection status
    if ((WiFiMulti.run() == WL_CONNECTED)) {
      outputsState = httpgrequest(serverName);
      Serial.println(outputsState);
      JSONVar myObject = JSON.parse(outputsState);
  
      // JSON.typeof(jsonVar) can be used to get the type of the var
      if (JSON.typeof(myObject) == "undefined") {
        Serial.println("Parsing input failed!");
        return;
      }
    
      Serial.print("JSON object = ");
      Serial.println(myObject);
    
      // myObject.keys() can be used to get an array of all the keys in the object
      JSONVar keys = myObject.keys();

      Serial.println(keys);
      Serial.println(myObject["active"]);
      Serial.println(keys[2]);

      int isActive = myObject["active"];

      if (isActive == 1) {
        digitalWrite(ledPin, HIGH);
      } else {
        digitalWrite(ledPin, LOW);
      }
      previousMillis = currentMillis;
    }
    else {
      Serial.println("WiFi Disconnected");
    }
  }
}

Is this even possible? I don't think you can redirect a POST/GET request from the client-side. This has to be done server-side i.e. the POST/GET request is processed by the server, and a separate connection created by the server to the ESP via some other means (web sockets, REST, etc)

yes it its.
a proof of concept, within LAN, therefore no port, but served on a different server / on a different machine

172.18.67.99 is my ESP
172.18.67.xx is my local "server" (its even just plain html file on my pc...)

<html>
<head>
<title>on a different webserver</title>
</head>
<body>

<form method="POST" action="http://172.18.67.99/cmd?cb=cboutputPin0&v=1">
	<input type="submit" value="On 0" name="B1" target='_self'></p>
</form>

<form method="POST" action="http://172.18.67.99/cmd?cb=cboutputPin0&v=0">
	<input type="submit" value="Off 0" name="B1" target='_self'></p>
</form>

<p>Alternative a simple href</p>
<p><a href=http://172.18.67.99/cmd?cb=cboutputPin0&v=0>on 0</a></p>
<p><a href=http://172.18.67.99/cmd?cb=cboutputPin0&v=1>off 0</a></p>
</body>

</html>

basically it doesn't matter from where you POST (or GET) the incoming request to the ESP as long as you can reach the ESP.