How to start programming with html code for esp8266 webservers ?

Projects Programming
1

Hi,

I came across the wifi area with the esp8266, and learned that I can embed html code within arduino ide code for the webserver UI.

So I want to learn how to design the webservers for the esp8266 boards.

My questions:

  1. Where to start learning html coding for esp8266 ? ebooks or websites.
  2. What is the best way to compile and debug the webservers for the esp8266. Within Arduino IDE or in another programming IDE ?
  3. Should I download Visual Studio and start writing/debugging html codes there ?
  4. Any other recommendations or suggestions ?

Thanks,

2

It's not clear if you already know HTML programming and are looking for advice about how to use it with an ESP8266

OR

If you don't know HTML and are looking for help with it.

If you don't already know how to create HTML then there are lots of websites with Tutorials. I usually use the w3schools website. I also suggest that you learn to create HTML on your PC frst as you can immediately view the results in your browser.

...R

3

Robin2:
It's not clear if you already know HTML programming and are looking for advice about how to use it with an ESP8266

OR

If you don't know HTML and are looking for help with it.

If you don't already know how to create HTML then there are lots of websites with Tutorials. I usually use the w3schools website. I also suggest that you learn to create HTML on your PC frst as you can immediately view the results in your browser.

...R

Actually the 2nd possibility which is that I don't know how to develop html.

I already viewed w3schools website it's really a nice clear tutoring area.

but there are things I found in arduino esp8266 code that I think a bit not in the basics of html and maybe specified for esp8266; like,

            // HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
            // and a content-type so the client knows what's coming, then a blank line:
            client.println("HTTP/1.1 200 OK");
            client.println("Content-type:text/html");
            client.println("Connection: close");
            client.println();

and this one:

if (header.indexOf("GET /2/on") >= 0) {

also this:

in this part the stuff here are different than the basic structure of html or maybe I have to dive more into html to understand the other parts of the attributes in each line:

            // Display the HTML web page
            client.println("<!DOCTYPE html><html>");
            client.println("<head><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">");
            client.println("<link rel=\"icon\" href=\"data:,\">");
            // CSS to style the on/off buttons 
            // Feel free to change the background-color and font-size attributes to fit your preferences
            client.println("<style>html { font-family: Helvetica; display: inline-block; margin: 0px auto; text-align: center;}");
            client.println(".button { background-color: #195B6A; border: none; color: white; padding: 16px 40px;");
            client.println("text-decoration: none; font-size: 30px; margin: 2px; cursor: pointer;}");
            client.println(".button2 {background-color: #77878A;}</style></head>");

and also other lines in that example of Rui Santos:

esp8266-web-server

4

wolfrose:
... and learned that I can embed html code within arduino ide code...

That's ok for a few lines for buttons etc.
For bigger projects (multiple/fancy web pages) it's easier to directly save the .html file to the ESP8266 module's built-in flash (SPIFFS), along with .js .css .png and other files.
Google "A Beginner's Guide to the ESP8266".
Leo..

5

Wawa:
That's ok for a few lines for buttons etc.
For bigger projects (multiple/fancy web pages) it's easier to directly save the .html file to the ESP8266 module's built-in flash (SPIFFS), along with .js .css .png and other files.
Google "A Beginner's Guide to the ESP8266".
Leo..

Thank you so much, this difference between putting few line within arduino code and for bigger html code to put it in built-in flash is a very important information for me to learn right now.

Also the memory structure of the esp8266 is also different to me, so you tell me that I can save the html file directly to the esp8266 flash, and then there's I guess no bootloader or OS to run this html file within esp8266, but what I understood that is the html file would run but the wifi connection ... but also what can the memory of the esp8266 hold of code ?

What is inside the esp8266 memory ? How does esp8266 run any code whether it is a c code, arduino code or html file.

6

Make another folder in the folder where your .ino file resides, and call it data.
All your .html .js .css .png .ico etc. files go in there.
Then install the LittleFS plugin for the IDE, which enables you to upload those files directly to the internal flash of the ESP8266 (separate process from uploading a sketch).
Most ESP8266 have 4MB of flash, which is plenty for many fancy dashboard-style web pages and/or sensor data.
Only the ESP-01 has limited (0.5MB) flash, but still enough.
Study all you can find on the net about LittleFS (newer/faster SPIFFS).
Then come back with questions.
Leo..

1 Like
7

as general advise, start with the IDE examples in the

ESP8266WebServer.

The ESP8266 comes with the ESP8266WebServer.h extension and there is no need to

  • set the HTML Header fields manually
  • struggle around with reading of parameters

if you are looking for third party tutorials, use the examples which are using

#include <ESP8266WebServer.h>

and send data with the .on method:

server**.on**("/", handleRoot);

Stay FAR away from the old style sources which add the header fields manually, all these start with something like

client.println("HTTP/1.1 200 OK");
client.println("Content-type:text/html");
client.println("Connection: close");
client.println();

and often have patterns like:

if (header.indexOf("GET /2/on") >= 0)

there is no need to do that manually on the ESP!

8

noiasca:
as general advise, start with the IDE examples in the

ESP8266WebServer.

I started with the example that set led on/off through the ip address web page. Then I checked the html lines in the code and knew how to modify the html lines.

The ESP8266 comes with the ESP8266WebServer.h extension and there is no need to

  • set the HTML Header fields manually
  • struggle around with reading of parameters
  1. Do you mean that I don't have to lean html programming when I want to design and web page for any project ?
  2. Which parameters you mean ?

Stay FAR away from the old style sources which add the header fields manually, all these start with something like

client.println("HTTP/1.1 200 OK");
client.println("Content-type:text/html");
client.println("Connection: close");
client.println();

and often have patterns like:

if (header.indexOf("GET /2/on") >= 0)

Actually this is the example I started with, and I thought it is the most convenient way right now is to embed html lines within arduino code or put the html file in data folder in the sketch folder.

But you mentioned that there is a more newer way, so it would be much better I guess than this one.

=======================================================
Now there are a lot of stuff that I don't know its limits. So I have to learn about some stuff about what can I do with the esp8266. For example; what you mean by:

there is no need to do that manually on the ESP!

Do you mean writing html codes ?

9
  1. Do you mean that I don't have to lean html programming when I want to design and web page for any project ?

No you do need to learn HTML, and particularly how to create a form.

  1. Which parameters you mean ?

The parameters that a user will include in either a post or get request (an ESP treats them nearly the same)

I thought it is the most convenient way right now is to embed html lines within arduino code or put the html file in data folder in the sketch folder

As he said, use the ESPwebserver instead. You create a callback in setup() and within that you can read the values of any parameters and create a response "HTML" webpage.

Do you mean writing html codes ?

Nope the HTML codes you do have to write, in fact, since there is no error checking on the HTML you create, you have to make sure that you create a proper head, body, and close all things you open ! (like paragraphs etc.)

1 Like
10

Wawa:
Make another folder in the folder where your .ino file resides, and call it data.
All your .html .js .css .png .ico etc. files go in there.
Then install the LittleFS plugin for the IDE, which enables you to upload those files directly to the internal flash of the ESP8266 (separate process from uploading a sketch).
Most ESP8266 have 4MB of flash, which is plenty for many fancy dashboard-style web pages and/or sensor data.
Only the ESP-01 has limited (0.5MB) flash, but still enough.
Study all you can find on the net about LittleFS (newer/faster SPIFFS).
Then come back with questions.
Leo..

Thank you,

You absolutely described in brief lines very important stuff that I really need to learn about right now. And knowing that I need time to search the net for information about LittleFS.

Also a very important information you mentioned is that the types of files I can upload to the flash of the esp8266.

I also need to learn about esp8266 memory system and structure.

For example, I learned that esp8266 doesn't have a program memory. I read in the datasheet that it has only SRAM and that's it. It's meant to work with an external flash memory.

But I have one question I want to know when people are working with SPIFFS. I learned that SPIFFS is a file system that uses hardware SPI to work with the external flash to put anything you want.

But my question, as I uploaded some sketches to the nodemcu without dealing with SPIFFS:

  1. Why would I need to use SPIFFS to store files on flash where I guess I can use Arduino IDE to upload any sketch to the ESP8266 ?
  2. Does Arduino IDE use a similar method to SPIFFS to upload any sketch in the examples to ESP8266 ? or what's going on exactly ?

I learned one bit, that one main advantage of using SPIFFS is that I can upload other codes other than Arduino C++ code; like, HTML and CSS in a separate folder alongside the arduino sketch. And of course there is a coding system that runs the HTML and CSS codes since the final working code to run ESP8266 microcontroller.

1 Like
11

wolfrose:
For example, I learned that esp8266 doesn't have a program memory. I read in the datasheet that it has only SRAM and that's it. It's meant to work with an external flash memory.

The modules (ESP-01, ESP-12) have that flash memory (small 8-pin chip), not the ESP8266 chip itself.
You will be using modules, so you have (4MB of) flash available.
The ESP8266 talks to the flash memory chip on the ESP-xx module with SPI (SPIFlashFileSysem).

Part of the flash holds your sketch, and the rest can be used as 'hard drive'.
As said, you must install a utility in the IDE to use the flash for files.
It's a separate process from uploading a sketch.
Maybe you should put your .html and .css files in the sketch for now,
but at some stage (larger projects) saving directly to flash is so much easier.
Leo..

12

You will be using modules, so you have (4MB of) flash available.

depending on the module you use, although even on an ESP-01, you could remove the flash chip that is on there and replace it with a bigger one.

  1. Why would I need to use SPIFFS to store files on flash where I guess I can use Arduino IDE to upload any sketch to the ESP8266 ?

The truth is you don't ! the advantages you've described already, the disadvantage is that you will need to be familiar with the SPIFFS and the library that you use to access it. You will still need to read those files, and send them to the client upon request. For small projects and even for pages that contain a lot of dynamic content it may not be very practical.

  1. Does Arduino IDE use a similar method to SPIFFS to upload any sketch in the examples to ESP8266 ? or what's going on exactly ?

Nope. The example sketches are simply in a specific folder on your harddrive (a sub folder of the library folder(s) ) and the IDE therefore lists them as examples (rather than part of your sketchbook) and disallows you from overwriting them. For the rest they are sketches just like any other. That examples show up for 1 board but not for another is simply because those libraries are board specific.

I learned that SPIFFS is a file system that uses hardware SPI to work with the external flash to put anything you want.

Technically speaking you could add more flash memory and access it by enabling/disabling it, but the libraries assume that you are using the primary flash chip.

1 Like
13

Wawa:
The modules (ESP-01, ESP-12) have that flash memory (small 8-pin chip), not the ESP8266 chip itself.
You will be using modules, so you have (4MB of) flash available.
The ESP8266 talks to the flash memory chip on the ESP-xx module with SPI (SPIFlashFileSysem).

Yep ! got that.

Part of the flash holds your sketch, and the rest can be used as 'hard drive'.

OK, so it's like the new PC HDD that doesn't have anything installed at first, then I install the windows OS and the rest of the HDD would be for anything else I want to put.

According to the following picture:

Are these 4 options of my preference or it's hardware related ?
For example; if I chose option 2 which is 4MB (1MB SPIFFS):
Does that mean Arduino would allocate 3MB for sketch code and 1MB for separate SPIFFS files ?

I'm following this tutorial:
randomnerdtutorials

As said, you must install a utility in the IDE to use the flash for files.
It's a separate process from uploading a sketch.

I just did that. And yeah I started to know what is going on.

  1. Upload the sketch with Arduino IDE.
  2. Upload the files in 'data' folder using this utility.

Maybe you should put your .html and .css files in the sketch for now,

Yeah you mean I would embed the html lines within arduino sketch and work on the process of uploading the separate files like html and image files later ?

but at some stage (larger projects) saving directly to flash is so much easier.

Got that thanks for the clarification. Really it's very important to me to understand the big picture of something before going into its details.

1 Like
14

Are these 4 options of my preference or it's hardware related ?
For example; if I chose option 2 which is 4MB (1MB SPIFFS):
Does that mean Arduino would allocate 3MB for sketch code and 1MB for separate SPIFFS files ?

For your preference, though keep in mind that if you flash with a SPIFFS size smaller than you had allocated before your data may be overwritten (not always, but you can not access it unless you have SPIFFS declared)

  1. Upload the sketch with Arduino IDE.
  2. Upload the files in 'data' folder using this utility.

Once you have the utility installed you should be able to upload from inside the IDE, there should be a new option in the 'Tools' called 'ESP8266 Sketch Data Upload' which will upload whatever is in the data folder of your current sketch.

1 Like
15

Deva_Rishi:
depending on the module you use, although even on an ESP-01, you could remove the flash chip that is on there and replace it with a bigger one.

Yeah meaning that esp8266 can work with any flash size, and SPIFSS support any flash. Of course knowing that SPIFFS is meant for small size systems.

I remember I read that SPIFFS support upto 128MB.

The truth is you don't ! the advantages you've described already, the disadvantage is that you will need to be familiar with the SPIFFS and the library that you use to access it.

found this about spiffs in instructables which outlines main points:

SPIFFS

Which stands for SPI Flash Filing System was designed for SPI flash devices on constrained embedded microprocessor systems with little RAM.

What SPIFFS does:

Specifically designed for low ram usage
Uses statically sized ram buffers, independent of number of files
Posix-like api: open, close, read, write, seek, stat, etc
Implements static wear leveling to prolong the life of system flash
What SPIFFS does not do:

SPIFFS does not support directories and uses a flat structure. Creating a file with path tmp/myfile.txt will create a file called tmp/myfile.txt instead of a myfile.txt under directory tmp.
It is not a real time stack. One write operation might take much longer than another. So is best used at start up, shut down or when time critical activities are not scheduled.
Presently, it does not detect or handle bad blocks.

===============================================================

You will still need to read those files, and send them to the client upon request. For small projects and even for pages that contain a lot of dynamic content it may not be very practical.

You mentioned really important points about spiffs, where should I use it and other project application type related restrictions/preferences.

so I think you mean by "dynamic content" is the variable content all the time. For example:
Whether station: in this type of application, esp8266 would read values from sensors and send them frequently to the http server so this is an example of "dynamic content", right ?

1 Like
16

so this is an example of "dynamic content", right ?

That is what i mean by it yes. There are more examples which come to mind of course. There is a way to actually let the client add the values to the page itself using HTML, but a more practical way is to add values to the HTML on the fly while creating the page from within the sketch. Of course you can do a combo of both. One thing to keep in mind is that if you are trying to save on RAM, the page anyway needs to be in RAM before it is being sent, so

void sendPage() {
  String s;
  s += " (.. the HTML code ..)";
  s += somevariable;
  s += " (.. more HTML ..)";
  server.send(200, "text/html", s);
}

Is as efficient in RAM use as having the whole page on SPIFFS, loading it into memory and sending it.
All literals are in progmem until assigned. The 'special characters' do present an issue, particularly the " since it is used a lot in HTML, so you lines would contain a lot of " 's. And the advantage of first creating a page (even using 3rd party software) and checking it in your browser before upload is not available.

17

The 'special characters' do present an issue, particularly the " since it is used a lot in HTML, so you lines would contain a lot of " 's.

Are single quotes, ie ', not acceptable for use in HTML in place of ", which avoids the need to escape the double quotes ?

18

Are single quotes, ie ', not acceptable for use in HTML in place of ", which avoids the need to escape the double quotes ?

Appears so yes ! oh i never even considered that. You do need to close with the same as with which you opened it seems. try it. This is quite a cute link btw. Your question provoked the search.

19

wolfrose:

  1. Do you mean that I don't have to lean html programming when I want to design and web page for any project ?
  2. Which parameters you mean ?

Ok. It's like this:

HTML stands for HyperText Markup Language. It's what web pages are written in - it allows you to describe a web page with images, formatted text, hyperlings, embedded javascript code - all that stuff. Everything between a <html> and a </html> is html.

HTTP stands for HyperText Transfer Protocol. It's a way of moving data from one place to another over the internet. It does this by the client and server sending a few lines of data to one another before sending the HTML itself.

HTTP concerns itself with URLs like: https://forum.arduino.cc/index.php?topic=705974.0, and with a bunch of sideband data that you don't normally see when you open a URL in a web browser.

So when your ESP8266 sends an html "hello world" to your web browser in reply to a GET message, what actually happens is that your web browser sends this over the wire:

GET /HelloWorld.html HTTP/1.1
Accept: text/html
Connection: close

And the web browser replies with:

HTTP/1.1 200 OK
Content-Type:text/html
Connection: close

<html><body>Hello, World!</body></html>

Of course, in a real request there will be much more than just the Accept and Connection headers.

As noiasca says, the ESP libraries contain stuff to make it far simpler to do the HTTP side of what a webserver needs to do - which can be very much more complicated than in this toy example here.

But even though the ESP takes care of much of it, it helps to understand what it is doing for you, especially if you want to do a complicated website with multiple pages and error handling.

You can work on just the HTML side of things by editing HTML files on your computer and opening them with a web browser. There's a world of stuff to learn, depending on how fancy you want your pages to be.

When it comes to serving those files and having the arduino do something useful when a user hits a button, you have to deal with HTTP. At the very least, you need to know that the stuff after a question mark in the URL is called the "parameters".

20

At the very least, you need to know that the stuff after a question mark in the URL is called the "parameters".

Or 'Arguments' Knowing what it's called is just part of it. If you want an inter-active webpage, you can create forms on the webpage, that once submitted, add user input to the URL (as arguments) and that can be extracted by the ESPwebserver using member functions like

.hasArg()
& .arg()

For many purposes where you will want to use an ESP to serve a webpage, you will want to use forms. Not always of course, sometimes you will just want a client to log in to view sensor data, and maybe you will want the webpage to automatically refresh. The rest is just navigation and cosmetics.

1 Like