I am trying to revive an old ESP8266 project that used the ESP8266 Webserver library, but on the allegedly superior ESP32. I converted the library includes from ESP8266Webeserver to ESP32Webserver and then wanted to convert the server.on statements from using inline web pages wrapped in consts to actual files on the LittleFS partition. The first problem I encountered was finding examples for ESP32Webserver. All Google searches and IDE examples seem to be based on ESPAsyncWebserver. The examples in the IDE do not include LittleFS usage.
The Async version is proving more trouble than it worth. Among other things the library has missing dependencies, but also requires me to use the ESPAsyncHTTPUpdateServer library for the OTA part, which has also proved problematic and has required a manual install.
I managed to get both libraries and dependencies in order when I then ran into a problem with the Webserver .arg() method which doesn't exist in the async version. For the ESP32 Async version, I found this, but I don't understand what it is doing.
The non-async version is straightforward by comparison, but I can't find examples for how to use LittleFS with the standard non-async Webserver library. I am assuming this is possible?
I would therefore appreciate some help with both of these problems. While a bit of an explanation of what the async code is doing would be appreciated, it would help me initially to understand how a server.on statement using the non-async version of the library with a file on LittleFS is formed. That would at least allow me to move forward and get the web pages uploaded and the concept tested. I can ponder over how to work with the async version in the longer term.
i find it important to use the correct set of libs that play nice together
this combo works for me
//xia esp32s3
Using library ESP Async WebServer at version 3.12.1 in folder: C:\Users\rc\Documents\Arduino\libraries\ESP_Async_WebServer
Using library FS at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\FS
β
Using library Async TCP at version 3.6.1 in folder: C:\Users\rc\Documents\Arduino\libraries\Async_TCP
Using library FauxmoESP at version 3.4.1 in folder: C:\Users\rc\Documents\Arduino\libraries\FauxmoESP
Using library WiFi at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\WiFi
Using library Networking at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\Network
Using library ArduinoOTA at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\ArduinoOTA
Using library Update at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\Update
Using library ESPmDNS at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\ESPmDNS
Using library LittleFS at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\LittleFS
Using library ArduinoJson at version 7.4.3 in folder: C:\Users\rc\Documents\Arduino\libraries\ArduinoJson
Using library ESP-FlexyStepper at version 1.5.1 in folder: C:\Users\rc\Documents\Arduino\libraries\ESP-FlexyStepper
Using library Preferences at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\Preferences
Using library Queue at version 2.0 in folder: C:\Users\rc\Documents\Arduino\libraries\Queue
Using library Hash at version 3.3.12 in folder: C:\Users\rc\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.3.12\libraries\Hash
*/
#include <ESPAsyncWebServer.h>
#include "fauxmoESP.h" //3.4.1 PAUL VINT
#include <FS.h>
#include <AsyncTCP.h>
#include <WiFi.h>
#include <ArduinoOTA.h>
#include <ESPmDNS.h>
#include "LittleFS.h"
#include <time.h>
#include <ArduinoJson.h> //7.4.3
#include "credentials.h"
#include <ESP_FlexyStepper.h>
#include <Preferences.h>
#include <cppQueue.h>
Thank you for that list. I can confirm that with a bit of help and research as well as some messing around I have the following installed and included:
Async TCP 3.6.1 (with spaces in name)
ESPAsyncHTTPUpdateServer 3.0.0
ESP Async Webserver 3.12.1 (with spaces in the name)
WiFi, LittleFS, ArduinoOTA and Networking are part of the ESP32 package which is at version is 3.3.12. I therefore think I do have the correct versions of everything.
I just need to understand how to use ESPWebserver with LittleFS and how to read POST arguments with ESPAsyncWebserver as it doesn't support the .arg() method.
This kind of reminds me of promises which are a nightmare to work with, but thank you for the example. If I recall, the values will print within the function where there serial.printf statement is commented out, but you can't access those values outside of the function. My guess is that if they were config values, they would need to be set inside the sreg function, likewise oif they were actions to turn on something, then that would also need to happen inside that function.
Can I ask what root[] and txb[] referr to in sreg()?
If you're talking about the <WebServer.h> built into the esp32 board platform, the initial WebServer example uses LittleFS
Looks like it's documented in the example's README(which you can't actually see from the IDE) which appears as a tab when the example is opened, along with the .ino
Did a search in the board package, and that example is the only one.
Don't know how you found that, but thank you for the link.
That makes sense. I arrived at the same conclusion literally just a couple of minutes ago. I use server.on() to bind the function call to the URL, but then have to use server.serveStatic() rather than server.send().
Unless I am mistaken, that will work fine for sending static data such as .js or .css files, but not for html pages with arguments where the data needs to be dynamically updated. My original code literally built html pages line by line in a char array buffer using individual strcat commands and inserting values into lines with sprintf. I see that examples still seem to do the same.
I wish there were a way to update the arguments on the page dynamically before sending it.
When i did it, i used Webserver.h for the ESP32, which is included in the core, and it translated into almost the same code for an ESP8266 & ESP32, but i didn't use the filesystem for generating a webpage though, so i am not usre about that part.
It's both: register a bunch of endpoints you want to handle manually, then have a fallback with static content. Given that the first argument is "/", it's the fallback for everything -- all request paths start with a slash.
And yeah, it's for files "on disk". You're not dynamically rewriting the file on LittleFS just to then pass it back statically.
When sending a response, the important thing is that the caller knows when you're done. That's either with a Content-Length header in the response -- "when you get this many bytes, it's done, no need to wait for more"; or with a chunked response, each of which has a length, and ends with a final zero-length chunk.
So you can build a single string, and when you're done, get its length to send before its bytes. But if you're building it piece-by-piece anyway, you can send those pieces over the wire as you have them. The latter is necessary to return large responses that don't fit in memory all at once.
The ChunkWriting example is pretty bare-bones
That chunkResponseBegin function takes a content type that defaults to text/plain. You could read chunks from a file or files on LittleFS and intersperse with dynamic content, without having to assemble a single response body.
BTW, one feature of the Async web server is templates, with a specific template processor function; as in the Templates example, which uses LittleFS.
This is one of the main reasons I avoid generating large dynamic HTML pages on the ESP32. A page can easily reach 50 KB or more, and the ESP32 may run out of memory because the full response often has to be assembled in RAM before it can be sent.
Instead, I keep large HTML files static on the file system and use JavaScript within those pages to fetch dynamic data through separate callback requests. Besides being much more memory-efficient, this approach also speeds up development and debugging because most changes can be made in the JavaScript without touching the HTML page itself.
Web development has sort of come full circle. In the beginning, no choice besides server-side rendering. Then JavaScript enabled client-side rendering, with the advantages you mention. But it also has its downsides: at least one additional request per page, and until that is fulfilled, the page is missing that info. So now the sweet spot is mostly SSR, with client-side hydration and enhancement. But that requires more sophisticated tooling and frameworks; perhaps not feasible on MCU, even on ESP32. (Sounds like a challenge.)
How often? It should be pretty rare that sending chunks won't work; only if while assembling the page, you need to tweak something that comes before. And in that case, if you're relying on JavaScript anyway, you can emit a bit of script at the end to tweak or insert items that show earlier in the HTML.
That may not outweigh the benefits of working with static HTML+JavaScript on the development and maintenance side. But SSR is generally better for the user experience (unless it's a true single-page app).
There is always something sent before the actual content, such as:
`Content-Length: xyz
The server is expected to send this header before the response body, but it cannot know the content length until the response has been fully generated, as you mentioned earlier. This often means the entire page must be kept in memory before anything can be sent.
There are workarounds, of course, such as using placeholders or generating the response in multiple passes, but I don't consider those particularly clean solutions.
Another advantage of the REST API + JavaScript approach is that it enables much more interactive web applications. This makes it easy to monitor and control the device directly from the web interface without reloading the page.
Exactly. I have been using it like that for years since the pages is serve tend to have a lot of dynamic content. I store up to something of 500 bytes in RAM and then send the chunk. I am still considering automating that data gathering and sending so when a certain is reached the chunk gets sent automatically, rather than me keeping an eye on it while writing, but it is not high on the priority list.,
I noticed earlier that the ArduinoJson library is being used, which is a convenient way to transfer multiple parameters at once. However, I am curious about this:
I looked up __attribute__ but still not sure what packed means and why it is being used here?
This definition:
makes me not much wiser. The only phrase that stood out here is "placed to minimize the memory required". But then there is this:
This particular point form that discussion seems poignant:
So if I arrange the struct appropriately from the largest type to the smallest, does this mean that __attribute__ ((packed)) is not actually needed?
In the example from post #4, it looks like the struct is being used to hold the incoming parameter data which is then being passed back in the JSON response. I think I understand the JSON code and am considering to use something like this in my project.
32 bit systems like esp32 build structures with each element starting on a 32bit aligned boundary in memory, the resulting structure is therefore padded out to align with those 32 bit boundaries. the structure is being sent to an 8bit system that expects no padding whatsoever. also when sending data serially all that padding is just wasted time .
so _attribute_ ((packed)) makes a structure of minimum size, that's compatible between different computer architectures .
json is a great way to transmit data in a human readable format between different system when speed and bandwidth are not limited.
my example takes data from a http GET request, formats it into a structure that is transmitted to foreign system via serial , waits to obtain the foreign response and then replies to the GET request with the foreign serial reply converted into a json object
in actuality the "GET" comes from a linux node-red system, is sent to an esp32 that connects to a victron solar regulator. so i can remotely interrogate and control a solar installation from anywhere