ESP-library fully interactive webserver that encapsulates the html-stuff

Hi Everybody,

I guess I have a rudimentary understanding of how an ESP-webserver works.

use library ESPAsyncWebServer.h to create a "server"-object

define what shall be sended as HML-code if a client sends a request to the ESP-Weberserver

  server.on("/sensor", HTTP_GET, [](AsyncWebServerRequest * request) {
    request->send_P(200, "text/html", myHTML_Code, processor);

where "processor" is the "replace placeholders with values" function

I'm absolutely not keen about the website's design any standard design will do for the website.

What I'm looking for is a library that reduces dealing with html myself to zero.
Yes you have read right zero. No messing around with a single character belonging to html.

This means this library should offer a framework with pure C++-functions where these c++functions do all the nescessary html-stuff in the backround.

This library should offer some simple functions to define relative positions on the website shown in the browser

print text at xy on the screen or print left / middle / right on the screen
if possible defining a font and fontsize

create a button with text "buttontext" on the button
create a inputfield of type integer, float min/max/step-values,
create text-inputfield

Full interactivity:

Whenever a value on the website has changed a callback-function specific to this input shall be called to instantly update variables inside the C++-code

second best thing would be a tutorial that explains how to create this kind of interactivity

best regards Stefan

So you prefer to learn a new API that can define a page layout in a rich way (will be complex) rather than learn the ubiquitous and standard HTML and CSS which is the outcome of decades of research and development?

If you want a crude web design there are very few HTML commands to know about. There are also web page design tools you could use to paint the page and get the associated HTML (see 15 HTML5 tools to make your life easier | Creative Bloq for example)

I would really recommend you take a small HTML tutorial - you’ll see it’s not that complicated to get going. Creating a fully adaptative and responsible page is more involved - not only on the HTML front. The server needs to handle it’s fair share of the work though

imho this makes not so much sense on a microcontroller with limited recources.
Furthermore I guess the time to "learn" the API of such a library will take at least the time to get some basic HTML and CSS knowledge.

this is what CSS was made for.
But as a spoiler, absolute positioning is not as good of you might think of. Have you ever heard of responsive design?

this is what html was made for.

You might consider a code generator where you offer a GUI / dashboard where the user can "design" his pages, places fields, assignes variables and genereates code and/or files which can be uploaded to the ESP. Hey why limit on ESP only, why not export code for Arduino :wink:

Hi @J-M-L Hi @noiasca
thank you for answering.

hm maybe I have to specify more precise what I mean.

There is a similar library for setup-options for what I would like to have with additional interactivity through specifying call-back-functions

esp-fs-webserver

Here is a picture of how the website looks
I have made the browser-window extra small to show how the html-code behaves in this case.
Reducing the with of the fields automatically

The code for setting up the website is as simple as this

  // Configure /setup page and start Web Server
  myWebServer.addOption("LED Pin", ledPin);
  myWebServer.addOption("A long var", longVar);
  myWebServer.addOption("A float var", floatVar, 0.0, 100.0, 0.01);
  myWebServer.addOption("A String var", stringVar.c_str());
  myWebServer.addOption("A bool var", boolVar);

on the float-var there are the options min / max / step

compared to my initial description this is reduced to a simple list lacking defining positions except the order in that the inputfields appear.
This is suffcient to me regarding the design-options.

This library does this for options to setup a device and then store the options in the flash-memory.

Understanding this "API" is a thing of ten minutes.

What I would like to have additionally is interactivity: which could be

  • click a "send back actual values" button that makes a callback-function run which updates variables in the code. One button for all fields

alternatively

  • individual buttons with individual callback-functions for each input field

alternatively

  • automatically call a callback-functions if a value is changed on the website to update the corresponding variable in the c++-code

  • a function-call from the c++-code to update a value shown on the website

learning this "API" will take another 20 minutes.

Now do you have a HTML-tutorial at hand that offers beeing able to understand the "API" within 30 minutes and having set up a demo-code in another 30 minutes that offers the same functionality ?

as there are:

  • on the website adjustable inputfields with options for min / max / step
  • boolean switches
  • interactivity as described above

If you know such a html-tutorial where I will be able to setup the same thing after 60 minutes post the link I will send you 20€ with paypal

I estimate each of you @J-M-L and @noiasca
have learned and worked a minimum of 300 hours about and with HTML-coding etc.
And I would not be astonished if it would be 3000 hours in summary.

Having learned so much and so long about html it seems to be a "piece of cake" for you. Because of this long time and big experience.

the esp-fs-webserver library offers even more:
uploading / editing files

and OTA updating.

The only point missing is interactivity like described above.

best regards Stefan

you are talking about GitHub - cotestatnt/esp-fs-webserver: From FSBrowser.ino example to an Arduino library (I guess).

after "10 minutes to learn the API" I have the impression:

  • I don't have the knowledge to write my own implementation now
  • this library seems not to be limited to "setup" only, at least the example gpio_list.ino shows how to activate/deactivate pins, or handleFormData.ino seems to gather data from the user.

Have you tried to get in contact with the writer of that library and just ask him about new features?

Yes I contacted user @cotestatnt And he added the min/max/step-options.
Maybe he has added even more. I have to lookup it new myself.
Thank you for pointing me to the examples

best regards Stefan

Hi @StefanL38
I fear that what you are asking is bordering on the impossible or in any case it requires so much C++ codind that it is not worth it.

To obtain the degree of interaction you require with the web page, it is necessary to make intensive use of web technologies such as Javascript and WebSocket.

As @noiasca pointed out, the example gpio_list.ino show a possible way to build a GUI which works in both direction: if a user set an output using the webpage the ESP handle the request and then update the actual state of output; the same thing occurs if an input pin change state from ESP side. This is done with the help of a WebSocket server and relative callback functions.

My library offers some "comforts" to develop your own webservers, but if you want to do complex and interactive projects you need to learn the basics of HTML, CSS but above all Javascript (which can be very similar to C ++ in syntax).

It may seem like a lot of work to do, but I assure you it is very satisfying and practically there are no limits to what you can do because your small ESP doesn't have to handle tons of C++ code but just stream some bytes to the web page which will take on all the heavy job (which run on a much more powerful PC or mobile device).

3 Likes

Hi @cotestatnt,

thank you for answering.

I have also used

and ESP-DASH pro which offers interactivity. But I'mnot satisfied with it.
The author will not develop ESP-DASH beyond what it is know

A longer time ago I took a short look into ESP-DASH. ESP-DASH uses VUE which seems to simplify things. But as I have zero experience with javascript, VUE, html etc. I didn't grasped it where I would have to modify things to what I would like to have to.

If you are experienced with javascript etc. you might be able or find it at least interesting to look "under the hood" of ESP-DASH

best regards Stefan

What about refreshing the website automatically?

Deep back in my head I remember that some years ago I discovered an option for the URL that forces the browser to repeat a real request to the webserver instead of using cached data.

But I don't remember it at all. I was using this with a little lua-script that created a tiny website. With this it was possible to update at least values measured by the microcontroller.

Can somebody say what option this is that must be added to the URL?

best regards Stefan

in the last century one would have used
meta tag refresh

I don't remember a standard function which would force a client to refresh the page by appending something to the URI.

today we use JavaScript, Fetch API, websocket,... to update values.

That is beautiful. But how does it work? And where do I find example-codes?

I tried to google it
far off
https://www.google.de/search?as_q=arduino+ESP+democode+update+values

not very successful
https://www.google.de/search?q=arduino+ESP+democode+Fetch+API

and "websocket" is such a general wide used term I expect thousands of hits but non of them specific to what I'm looking for.

Would you mind to post more specific words or links to tutorials that show that?

best regards Stefan

It's not a tutorial but I have a basic but comprehensive example of a webserver which does

  • switch pins
  • read pins
  • update values (via JavaScript and Fetch API)
  • has a CSS with some rudimentary responsive design.

As mentioned, I don't claim it to be a tutorial, but the sketch is explained in such a way, how I would start with a webserver.

https://werner.rothschopf.net/microcontroller/202108_esp_generic_webserver_en.htm

I don't have prepared information about websocket.

Hi @noiasca ,

thank you very much for your demo-code.

I compiled the demo-code and it seems to work.
Though I have some questions.

I started analysing the code
The esp_webserver_generic-tab define

#define USE_BOARD 65                   // the actual board to compile:

If I understand right this means
the tab config65.h is used

inside the config65.h-tab

two arrays are defined

// assign pins to arrays:
Pin inputPin[] {
  // the pin    a name for the website
  {inputAPin,   "Optocoupler"},         
  {flashPin,    "Flash Jumper"},
};

OutputPin outputPin[] {
  // the pin     a name    LOW active (or high active)
  {ouputAPin,   "Relay",   LOW},
};

From this declaring I would expect that the website would show
two inputpins and one output
but the website shows only the output-pin

So what am I missing that the defined inputpins were not shown?

edit: OK I found it. it is on page1

best regards Stefan

what in your code determines the update-speed of the input/outputs?
The time between clicking the button and updating the website is 4 to 5 seconds.

Is there a way to speed this up to "almost instantly" ?

best regards Stefan

line 94 in main tab.

const uint16_t ajaxIntervall = 5;      // intervall for AJAX or fetch API call of website in seconds

try 1 instead
(don't use 0 because it gets multiplied by 1000 in the server tab ... if 1 sec is to slow, multiply with 100 instead ...)

P.S.: it's not AJAX (anymore) and the english/german typo might be changed later :wink:

OK - this is an important point.

Also - are you saying that you want only one unique form with fields on top of each other and one unique submit button at the bottom ➜ this is of course simpler.

if you want to be able to place multiple buttons, with precise control on where they appear, each having its own callback and multiple value fields and being able to submit/update each field (refresh the page or just one field when a variable changes on the server for example), then you start entering into more complexity.

Yes. The main thing is to have an interactive webinterface at all.
Just one big list with all options and a single submit / safe-button.

Everything on top of that is nice to have but not crucial.
best regards Stefan

would the browser have access to the internet (separately from the Arduino)?

if yes, you could use some javascript classes for drawing the elements, jQuery UI for example which makes it somwhat simpler to design your web page (I understand this is not what you expect but remain convinced this brings much joy when you can design exactly what you want :slight_smile: )

Yes the browsers have access to the internet.
I have seen somewhere such a thing. There it seemed to me that some online-ressources were used for the design of the website.
If this is the case the code would depend on these additional things beeing online available over time.

I prefer to store everything locally. Even if this means that the design becomes very rudimentary and simple. The main thing is to have an interactive webinterface at all.
If it is easy to recognise this is an inputfield / this is a boolean on/off-option it is sufficient.
If everything is stored locally it will still work even if the online-ressources were gone (into history)

If I would like to have large HTML-code in some projects I could add an SD-Card.

best regards Stefan

So I gave it a quick try after dinner, here is something you can start with and expand on

This is built on top of ESPAsyncWebserver so you need to install that library

test.ino

#ifdef ESP32
#include <WiFi.h>
#include <AsyncTCP.h>
#elif defined(ESP8266)
#include <ESP8266WiFi.h>
#include <ESPAsyncTCP.h>
#endif

#include <ESPAsyncWebServer.h>
AsyncWebServer server(80);

#include "htmlConfig.h"

const char* ssid     = "********";
const char* password = "********"; 

void notFound(AsyncWebServerRequest *request) {
  Serial.printf("notFound: %s\n", request->url().c_str());
  request->send(404, "text/plain", "Not found");
}

void handleConfig(AsyncWebServerRequest *request) {
  int args = request->args();
  for (int i = 0; i < args; i++) {
    Serial.printf("%s -> %s\n", request->argName(i).c_str(), request->arg(i).c_str());
  }
  request->send(200);
}

void configPage(AsyncWebServerRequest *request) {
  AsyncResponseStream *response = request->beginResponseStream("text/html");
  startResponse(response);
  addRange(response, "Range", "range0", 20, 120, 1, 40);
  addNumber(response, "Number", "number0", -20, 20, 0);
  addCheckbox(response, "Checkbox 0", "ckbox0", true);
  addCheckbox(response, "Checkbox 1", "ckbox1", false);
  addButton(response, "Button", "btn0", "Click Me!");
  addNiceButton(response, "Nicer Button", "btn1", "Arduino");
  addRadioButtons(response, "Radios", "radio0", 1, 3, "radio 0", "radio 1", "radio 2");
  endResponse(response);
  request->send(response);
}

void setup() {
  Serial.begin(115200); Serial.println();

  WiFi.disconnect();

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  if (WiFi.waitForConnectResult() != WL_CONNECTED) {
    Serial.printf("WiFi Failed!\n");
    return;
  }

  Serial.print("IP Address: ");
  Serial.println(WiFi.localIP());


  server.on("/", HTTP_GET, configPage);
  server.on("/update", HTTP_GET, handleConfig);
  server.onNotFound(notFound);
  server.begin();
}

void loop() {}

htmlConfig.h

#ifndef _HTMLCONFIG_
#define _HTMLCONFIG_

#include <ESPAsyncWebServer.h>

void startResponse(AsyncResponseStream * response);
void endResponse(AsyncResponseStream * response);

void addRange(AsyncResponseStream * response, const char * label, const char * name, int minV, int maxV, int step, int value);
void addNumber(AsyncResponseStream * response, const char * label, const char * name, int minV, int maxV, int value);
void addCheckbox(AsyncResponseStream * response, const char * label, const char * name, bool checked);
void addButton(AsyncResponseStream * response, const char * label, const char * name, const char * text);
void addNiceButton(AsyncResponseStream * response, const char * label, const char * name, const char * text);
void addRadioButtons(AsyncResponseStream * response, const char * label, const char * name, uint8_t selected, uint8_t count,  ... );

#endif

htmlConfig.cpp

#include "htmlConfig.h"
#include <cstdarg>

const char configStart[] PROGMEM = R"--(
<!doctype html>
<html lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
  <title>Configuration</title>
<style>
input,output{display: inline-block; vertical-align: middle;}
th, td{text-align: left; padding: 5px; font-weight: normal;}
.button {
  background-color: white; 
  color: black; 
  border: 2px solid #377F83;
  border-radius: 10px;
  padding: 5px 10px;
  text-align: center;
  text-decoration: none;
  display: inline-block;
  transition-duration: 0.4s;
  cursor: pointer;
}
.button:hover {
  background-color: #377F83;
}
</style>
</head>
<body>
<table>
)--";

const char configEnd[] PROGMEM = R"--(
</table>
<script>
    function updateValue(N, val) {document.getElementsByName(N)[0].value=val;}
    
    function callback(N, val) {
      var req = new XMLHttpRequest();
      url = "/update?" + N 
      if (val != null) url += "=" + val
      req.open("GET", url , false); 
      req.send(null);
    }
</script>
)--";

void startResponse(AsyncResponseStream * response) {
  response->printf(configStart);
}

void endResponse(AsyncResponseStream * response) {
  response->printf(configEnd);
}

void addRange(AsyncResponseStream * response, const char * label, const char * name, int minV, int maxV, int step, int value) {
  response->printf("<tr><th>%s</th><th><input type=\"range\" name=\"%s\" min=\"%d\" max=\"%d\" step=\"%d\" value=\"%d\" data-show-value=\"true\" oninput=\"updateValue(this.name + 'V', this.value);\" onchange=\"callback(this.name, this.value);\"><output name=\"%sV\">%d</output></th></tr>",
                   label, name, minV, maxV, step, value, name, value);
}

void addNumber(AsyncResponseStream * response, const char * label, const char * name, int minV, int maxV, int value) {
  response->printf("<tr><th>%s</th><th><input type=\"number\" name=\"%s\" min=\"%d\" max=\"%d\" value=\"%d\" onchange=\"callback(this.name,this.value);\"></th></tr>",
                   label, name, minV, maxV, value);
}

void addCheckbox(AsyncResponseStream * response, const char * label, const char * name, bool checked) {
  response->printf("<tr><th>%s</th><th><input type=\"checkbox\" name=\"%s\" %s onchange=\"callback(this.name, this.checked ? '1' : '0');\"></th></tr>",
                   label, name, checked ? "checked" : "");
}

void addButton(AsyncResponseStream * response, const char * label, const char * name, const char * text) {
  response->printf("<tr><th>%s</th><th><input type=\"button\" name=\"%s\" value=\"%s\" onclick=\"callback(this.name, null);\"></th></tr>",
                   label, name, text);
}

void addNiceButton(AsyncResponseStream * response, const char * label, const char * name, const char * text) {
  response->printf("<tr><th>%s</th><th><button class=\"button\" name=\"%s\" onclick=\"callback(this.name, null);\">%s</button></th></tr>",
                   label, name, text);
}

void addRadioButtons(AsyncResponseStream * response, const char * label, const char * name, uint8_t selected, uint8_t count,  ... ) {  // would be better with template and recursive variadic function
  va_list vl;
  va_start(vl, count);
  response->printf("<tr><th>%s</th><th>", label);
  for (uint8_t i = 0; i < count; i++) {
    const char * radioItem = va_arg(vl, const char*);
    response->printf("<div><input type=\"radio\" name=\"%s\" value=\"%u\" %s onchange=\"callback(this.name,this.value);\">%s</div>\n",
                     name, i, (i == selected) ? "checked" : "", radioItem);
  }
  response->printf("</th></tr>");
}

You don't have to do anything with the .cpp and .h file, just make sure the .h is included in your .ino ➜ They are utility functions (of course both files needs to be in your sketch’s directory. feel free to make these utility functions into a library then a simple #include <htmlConfig.h> would bring you the capability).

The way it works is that you implement a page call back that will paint a new page in your web browser, the standard way for the ESPAsyncWebserver library.

here for example you'll see that I'm saying
server.on("/", HTTP_GET, configPage);
➜ if a browser request the home page (/) then the configPage() function is called. It's the responsibility of that function to provide an HTML answer that will be used to draw the page.

In the implementation of that function you have access to helper functions

the body of that function needs to look like this

void configPage(AsyncWebServerRequest *request) {
  AsyncResponseStream *response = request->beginResponseStream("text/html");
  startResponse(response);

  // HERE YOU ADD YOUR ELEMENTS

  endResponse(response);
  request->send(response);
}

so what elements can you add ?

I've implemented:

  • range (a slider with min, max, step) ➜ addRange()
  • number (a chooser for a number between min and max) ➜ `addNumber()
  • check boxe (with the option to be checked or unchecked at start) ➜ `addCheckbox()
  • two types of buttons (one very plain and one that looks better) ➜ addButton() and addNiceButton()
  • radio buttons (as a column of choices) ➜ addRadioButtons()

hopefully the parameters of those functions are self explanatory and entering this description

In the code I have

void configPage(AsyncWebServerRequest *request) {
  AsyncResponseStream *response = request->beginResponseStream("text/html");
  startResponse(response);
  addRange(response, "Range", "range0", 20, 120, 1, 40);
  addNumber(response, "Number", "number0", -20, 20, 0);
  addCheckbox(response, "Checkbox 0", "ckbox0", true);
  addCheckbox(response, "Checkbox 1", "ckbox1", false);
  addButton(response, "Button", "btn0", "Click Me!");
  addNiceButton(response, "Nicer Button", "btn1", "Arduino");
  addRadioButtons(response, "Radios", "radio0", 1, 3, "radio 0", "radio 1", "radio 2");
  endResponse(response);
  request->send(response);
}

Which draws that page:

So that's the "page design" phase. How does the user interaction work ?

All items you add on the page will trigger an HTTP GET upon change for /update with a parameter ➜ so if you declare a call back for /update like this

  server.on("/update", HTTP_GET, handleConfig);

then the handleConfig() function is called.

That's the function where you need to implement you business logic ( based on the parameters you get for /update you decide what to do).

you can see I provided a standard function for this

void handleConfig(AsyncWebServerRequest *request) {
  int args = request->args();
  for (int i = 0; i < args; i++) {
    Serial.printf("%s -> %s\n", request->argName(i).c_str(), request->arg(i).c_str());
  }
  request->send(200);
}

this only prints the parameters you get when the function is called. what you get is the name of your widget and if there is a value attached you get the selected value as well.

for example:

For the range, if you move the cursor to 76 the function will print range0 -> 76

For the number, if you go up one notch, the function will print number0 -> 1

For the first checkbox, if you unselect it, the function will print ckbox0 -> 0

For the second checkbox, if you select it, the function will print ckbox1 -> 1

for the plain button, if you click it, the function will print btn0 -> (there is no parameter)

for the nice button, if you click it, the function will print btn1 -> (there is no parameter)

last but not least, for the radio buttons, when you select one you get its position in the list (starting at 0). for example if you select "radio 1" you'll see radio0 -> 1

so, whilst I did not push it all the way to have a structure describing the page with the call backs, you have the tools to build that yourself if you want. At the moment to get the call back you would just have to do (as there is only one arg)

void handleConfig2(AsyncWebServerRequest *request) {

  if (strcmp(request->argName(0).c_str(), "range0") == 0) {
    // the new value is in string you can access with request->arg(i).c_str()

  } else if (strcmp(request->argName(0).c_str(), "number0") == 0) {
    // the new value is in string you can access with request->arg(i).c_str()

  } else if (strcmp(request->argName(0).c_str(), "ckbox0") == 0) {
    // the status is in string you can access with request->arg(i).c_str(). "1" is checked, "0" is uncheked

  } else if (strcmp(request->argName(0).c_str(), "ckbox1") == 0) {
    // the status is in string you can access with request->arg(i).c_str(). "1" is checked, "0" is uncheked

  } else if (strcmp(request->argName(0).c_str(), "btn0") == 0) {
    // there is no parameter, the button has been pressed

  } else if (strcmp(request->argName(0).c_str(), "btn1") == 0) {
    // there is no parameter, the button has been pressed

  } else if (strcmp(request->argName(0).c_str(), "radio0") == 0) {
    // the selected item index (starting at 0) is in string you can access with request->arg(i).c_str().

  } else {
    // unknown
  }
  request->send(200); // don't forget to return 200 to the web browser to say all went well
}

hope this make sense and helps a bit.

PS/ make sure to use different names for all the widgets.