Recommendation for a tutorial showing the pattern how to code webinterfaces with ESPAsyncServer

Using Arduino Programming Questions
1

Hello everybody,

I have only a very small knowledge about webservers and html-coding in general and how writing code to create interaction between an ESP32, connected to a WiFi-network, is done.

inspired by another users posting I want to ask if somebody knows of a tutorial that shows the

basic pattern

how to code it.

Of course there are hundreds of tutorials online that show one particular project providing a full blown code for this project but do almot not explain the basic patterns behind it.

I mean this
If you want to have async-webserver functionality the code to implement that is

// piece of code that shows how to **setup** up the asyncWebserver

And then has following up chapters that show

  • to create a titeled checkbox in the webinterface you need to write this html-code...- to "transfer" the checkboxes state into a variable of your arduino-code you write this ....

  • to create text-input-field in the webinterface you need to write this html-code...

  • to "transfer" the content into a variable of your arduino-code you write ....

  • to create a button "save parameters" in the webinterface you need to write this html-code...

  • to "signalise" button has been clicked to the your arduino-code you write ....

etc.

ideally with 3 or 4 examples that vary the describing text in the webinterface and vary variable-names to make it easier to recognize the pattern behind the examples.
Once the pattern is understood it becomes much easier to adapt the given demo-codes to the users own needs.

The description above might have some misconceptions about how interacting with webinterfaces works. Well from this you cen see that the tutorial should be start at the very basics.

best regards Stefan

2

I've written a couple intro tutorials in french (Les Bases d'un Serveur Web sur ESP-01 en commande AT and Techniques "avancées" de serveur web sur ESP8266). May be google translate can help you read those.

I think there are important things to understand before diving into a web code though:

If I want to be simplistic, the way it works could be described as follow:

  • The client (basically a web browser) opens a connexion to the server and sends a request using the HTTP protocol (basically you type a URL to get things started)

  • the server analyzes the request and sends an answer in the form of a document using a normalized description language (basically HTML, CSS, Javascript).

  • the client interprets the document description and paints the screen / acts accordingly

At that stage the connexion is usually closed. The request has been handled. There is no more link between the client and the server.

Going back to the server means you do that cycle again: the client contacts the server over HTTP and requests a new document.

Where the magic happens is that:

  • In the page description language you have specific capabilities (basically forms with textfield, checkbox, buttons, ...) to describe an interactive User Interface and what happens if the user on the client triggers an action (basically this is a request for a new document from the server and some data from the web browser will be sent to the server).

  • the HTTP protocol describes various Request Methods and the two most common are GET and POST. In particular they describe how the data (status of your check box, content of a textfield, ...) is encoded within the Request Method.

  • The server recognizes the Request Method being used, decodes the parameters and takes action to build a new document that is sent back to the client

  • the connexion is closed and the cycle can start over again.

There is one more thing:

  • In the page description language You also have the capability to provide some code written using a programming language (Javascript) that is interpreted by the client.
  • The programming language can affect how things are painted over the screen
  • One specific possibility is also to trigger an HTTP request and get an answer: a frequently used technic is called AJAX. With AJAX you can read data from a web server after the page has loaded, you can update a web page without reloading the page and you can send data to a web server - in the background.

Libraries such as ESPAsyncWebServer makes it easy to set up a server ready to handle any form of Request Methods. The library will help you unpack the parameters and provide various ways of sending the document answer back to the client (dealing with the necessary protocol information so that you can only focus on the description).

Once you've said this, it becomes evident that you won't do anything relevant if you don't invest a bit of time in understanding the underlying document object model for a web page and HTML, CSS and Javascript because that's fundamentally what drives the interaction. If you can't write the document then you can't prime that pump.

➜ It depends on the learner how they want to achieve this.

I started looking at this a long time ago when HTTP was being developed and I followed along all those years, out of curiosity mainly, to stay aware of the new capabilities. So I learnt about the HTTP protocol, then HTML, CSS and about Javascript but at the time I was exploring this, there was no ESP so I did not test anything along the way.

If I had to learn now, that's how I would do it, it suits my learning style (may be not suited for everyone) as
I learn better by understanding all the building blocks and concepts, independently of of each others and then putting things together.

You have tons of training material available on line and you can go as deep as you want on each.

1/ Read a bit about HTTP and play with your ESP to set up a basic server listening on port 80 and see what you get when you type an URL in your favorite web browser

2/ Read a bit about HTML and generate a simple page that you send back to the client. As everything is text, you can play around in your ESP code to dynamically generate HTML (eg send the value you read on A0 for example).

3/ Then read about Javascript and add some auto update capability to your page. Every second you request an update to that page, so you get the page re-painted with the value of A0 every second.

4/ Then keep exploring HTML and forms. Build a page with a form and a submit button and see what you get on your ESP when you click submit.

5/ Then read about the DOM (Document Object Model) and how Javascript can be used to dynamically access/read/write the elements on your web page.

6/ Then Explore AJAX and start playing with it still with your ultra simple ESP listening on port 80.

At that stage you start understanding that extracting information from the request is really about text parsing. Having a good grasp on cStrings functions helps or you can use the String class.

If you got that far, you have accumulated enough experience and understanding on the pieces of the puzzle and that's where Libraries such as ESPAsyncWebServer can come into play to avoid re-inventing the wheel.

Libraries are specific and then you can read tutorials and examples from those libraries. But as you are now equipped with the concepts, you know what's going on in the background and there is no black magic.

My strong belief is that if you lack the foundations, you are building on sand and moving grounds and can't really do something great, you'll be trying all sorts of things and workaround because you don't get what's happening behind the scene. (you can always try to work your way up from sample code but I don't find this satisfying).

3

Hi J-M-L,

thank you very much for your repsonse. It helped me understand a bit more how it works.
Following your suggested path will surely give a pretty deep insight how it all works.

I want to do a comparison with another library:
If I use a libraries like

#include <OneWire.h>
#include <DallasTemperature.h>

or

#include <FastLED.h>

I can do this "on the surface" with which I mean I use the functions provided by the library

DallasTemperature sensors(&oneWire);

  sensors.begin();
  deviceCount = sensors.getDeviceCount();
  sensors.requestTemperatures();

without looking "under the hood" how all the bitbanging on that one wire of the onewire-interface has to be done.

If I would like to have some special kind of function that reads two onewire-sensors with the smallest 48-bit device-ID and the biggest device-ID I would have to dive into the details of the library to add such a special function. Diving into into library offers changing all details and all functions to maybe special needs.

I see it similar with html, css, javascript. I could dive into the fundamental details of html, css and javascript and after learning this all
having a quite profund knowledge to change a lot of details of how the webinterface looks like.

I have made the very conscise decision to "stay on the surface"
which means being NOT able to change all the details and sometimes to stumble across "why does it not work in this variation?"

So what I'm looking for is some wrapper-functions like in this sketchy description

String createHTML_Title(String myHeaderText)
String createCSS_Part(String NameOfFont, TextColor)

String createButton(String ButtonText)
String createTextEditField(String TextReturned, String FieldTitle);
String CreateSlider(int min, int max, String Title)

To get a very simple single-columm website for showing some status-information and entering parameters

leaving away all the fancy stuff like multicolumm, Buttoncolor,
defining different textsizes on everything, "onMouseover popups" etc. etc.

The goal is to have a set of arduino-functions where my work is reduced to write the code
with the characters what is the meaning of each element (like button, input-field, slider, statustext)

and easy to use functions to take over user-data into variables that was entered into the website

I'm very aware of that this approach limits the possabilities to adapt certain details. These limitations are the price for beeing able to setup such a webinterface within 2 or 3 hours instead of learning 120 to 230 hours how html, css and javascript works in detail.

best regards Stefan

4

I understand what you say but the abstraction provided by the library are not the same:

  • DallasTemperature or FastLED offer you an abstraction of the device and let you drive the device indeed without having a need to understand what's happening in the background

  • ESPAsyncWebServer is the abstraction for the WebServer part and HTTP concepts. So you don't have to understand the HTTP protocol, how parameters are encoded, etc but It has very limited support for what relates to what the user sees, the document itself (offers the ability to replace place holders in HTML by dynamically generated content when you build an answer).

So it seems to me that what you are asking for is a library for generating a document. You could have a look there and see if that meet your needs:

(not sure it deals with dynamic stuff)

5

what I was saying above does not require 120h as you build up skills as you go. You don't need CSS for example and can ignore also javascript if you want a simple, form driven interface. (some fields, a button, you click and it gets submitted)

6

J-M-L has already covered lot of topics. So my answer might not give a lot of additional input.

Anyhow...

I think the real challange to this topic is that "a webserver" is a quite huge topic. The "problem" for most people is not only the right syntax or usage of the (async) Webserver API but the lack of knowledge how things work together.

My impression is only the minority want to read basic explanations. Therefore a "full" project showing one usecase fits more readers.

to extend your list, imho one of the most asked questions is "how to update values on a page" (like your last bullet), so you should cover FetchAPI/AJAX, JavaScript, JSON. Difference between GET and POST and when to use what, how to read and process parameters, how to read HTTP-Header fields. Oh and at least some CSS should be added also to the list (otherwise people are fidling around with HTML tags to "design" their entry forms...).

The challange is, "none" of the topics are related to the async webserver, but you might need all of them to make a webpage working in a 2021 style...

7

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.