HTML client.read response handling

Projects Programming
1

Many Adruino example simple server programs use a client.read to determine if the web page on/off HTML button was clicked. If clicked the results from a GET request is searched for a particular keyword, for example, as follows "/LED=ON"... Then the sketch turns on the LED.

Typically the get request returns the client DNS name (or IP address) followed by a payload containing the aforementioned keyword (/LED=ON).

Okay, all good so far. The problem is that if the browser web page is simply refreshed the same GET results is again returned causing the server to think the same button was clicked again. This behavior is obviously undesirable and I'm looking for an easy solution.

I am amazed that virtually every similar-function example program seems to have the above shortcoming. Kinda makes me think I'm doing something wrong.

Here is one such program Controlling an LED over Wi-Fi with ESP8266 and Arduino | by Vedant M | Medium

I have virtually no HTML knowledge so please keep it simple.

Thanks, Frank

2

You should probably use the POST method rather than GET method. Or, rather, use POST in addition to GET.

That way, the browser on your phone/laptop will send a GET request to the Arduino when the page is first opened or refreshed. It will send a POST request when a button on the page is pressed.

This will prevent a refresh of the web page causing unwanted effects.

Your code will need to change to separately handle the GET and POST requests. It should respond to the GET requests by returning the HTML for the page, as it does now. It should respond to the POST request by actioning the button press and returning a success/failure message.

When using POST, the parameters are sent in the body of the request, not as part of the URL like they are in GET requests.

No, they are. Because it's easier. But it has the downsides you have noticed.

3

PaulRB, Thank you.

Problem is I am not explicitly doing a GET request I'm doing a client.read(). Is there a similar function for doing a PUT?

I suspect making this change to a PUT may involve a little more than simply changing one statement (client.read to say client.put). If so, is there a simpler workaround?

Thanks Again
Frank

4

The client, your browser, is doing the GET request. The server, your Arduino, would use client.read() for either GET or POST.

The client, your browser, won't be doing a PUT. Don't confuse POST and PUT, they are different methods.

1 Like
5

Oops, right you are, my bad. I inadvertently reverted to a prior life experience.

I'll have to recheck Google using POST.

6

In HTML, <a href= is the start of a link. Clicking it is the same as using a bookmark, or typing the URL directly: it makes a request using the HTTP method/verb GET, effectively the default method.

If you follow that (mediocre) linked tutorial

  String request = client.readStringUntil('\r');
  Serial.println(request);

will first print (without the handy syntax highlighting shown here)

GET / HTTP/1.1

for the initial page load, and then

GET /LED=ON HTTP/1.1

when you click the ON button. This first line of the request is the start-line or request-line, comprising

  • the method/verb
  • URI-path, always starting with a slash
  • HTTP version

The code blindly searches

if (request.indexOf("/LED=ON") != -1) {

the line, meaning it would also turn on if you request

DELETE /blah/LED=ONyx HTTP/1.1

but for a (mediocre) tutorial... eh. More egregious is representing the same web page resource as three different paths.

For the web page to do a POST instead, the quickest/simplest way is with a form

<form method="POST">
  <button name="LED" value="ON">ON</button>
  <button name="LED" value="OFF">OFF</button>
</form>

However, this has a drawback: now if you refresh, the browser will prompt you to "Confirm Form Resubmission". This is because the current page is the result of a POST, and that might be for something like making a purchase. You wouldn't want a refresh to buy something again. Also the prompt may just be plain confusing.

The solution is with the Post/Redirect/Get pattern:

  1. POST the request that changes something (that you don't want repeated with a refresh)
  2. The server does not respond directly (with an HTML page) but instead returns a redirect, commanding the browser to load something else
  3. That something else (in this case) would be the same page originally requested with GET, which is safe to load repeatedly, and would now show the updated status

This also clearly separates the kind of request for a given resource, which in the tutorial, is the root page / -- perfectly fine for a simple Arduino controller page

  • GET returns the status, and includes buttons to change things
    • response 200 OK with HTML
  • POST changes the status
    • response 303 See Other with the Location header

If you have exactly one page, then you're no longer parsing/searching the URI-path; you can act solely upon the method/verb, and by parsing the POST body.

1 Like
7

Kenb4, thank you for taking the time for such a detailed answer. I got the basic idea of your answer, but I'll have to study it before I can implement it. That may take me some time.

Further all the responses have inspired me to think of another solution as follows:
Upon receiving a valid "/LED=ON" response from the server's client.read() command and turning on the LED (or whatever) the next step is to repaint the client's screen. Instead of repainting the original screen I'll slightly modify the button response to be say "/LED=ON2". And of course the server receive logic will then have to check for "/LED=ON2" as the valid ON command. So, any subsequent "/LED=ON" reply by a refresh would then be ignored.

Basically the server would alternate the button parameters upon screen repaint between "/LED=ON2" and "/LED=ON" each time a valid response is received.

It sounds simple, but I haven't tried to code it yet thinking somebody on this forum may spot a major flaw.

Thanks, again, everyone for the great help.
Frank

8

First of all, requesting to turn on an LED that is already on doesn't seem like much of an issue. I suppose this is a stand-in for what you're actually trying to do, and to avoid repeating with a refresh?

Alternating ON and ON2 is susceptible to the Back button:

  1. initial load (requires ON)
  2. click ON link (now requires ON2)
  3. click ON2 link (now requires ON)
  4. Back
  5. Refresh

Requiring POST to change something, and preventing inadvertent replay of those -- which the browser tries to help with -- is more reliable. Modifying code like that tutorial is not too complicated. The HTML is the same; the difference is replacing the part that checks for the change request

if (request.startsWith("POST")) {
  // read request body with `client` and make changes; then
  client.println("HTTP/1.1 303 See Other");
  client.println("Location: /");
  client.println();
} else if (!request.startsWith("GET")) {
  client.println("HTTP/1.1 405 Method Not Allowed");
  client.println("Allow: GET, POST");
  client.println();
} else {  // GET
  client.println("HTTP/1.1 200 OK");
  client.println("Content-Type: text/html");
  // send HTML like before (or better)
}
9

Wikipedia gives a good overview what is happening and what the PRG pattern should do.

It was already on my todo list and I have now updated my simple webserver example with such POST -> REDIRED -> GET directive now. It should work without the retransmission warning of the server.

Furthermore the example implements a state machine to process the incoming HTTP request and should work stable with GET and POST parameters and scope with the favicon.ico request from most browsers.

Important:
My example should only be used for the basic webclients like Arduino UNO R3/NANO/MEGA.
If you write a program for an ESP8266/ESP32 use the ESP8266WebServer/HTTPWebServer instead! (!!!)

What microcontroller are you using?

10

kenb4, thanks for recognizing the flaw of my solution and thanks for the additional code.

Yes, I now see where the back button followed by a refresh would cause the same problem. Although I already implemented my solution and I think I can live with that flaw as I believe that series of keystrokes would be rare.

On any future project I'll try and implement your solution.

Thanks, again
Frank

11

noiasca,
Thank you..

I am using an esp8266, thanks for the related tip.
Frank

12

when I see a

client.println("HTTP/1.1 200 OK");

in an ESP8266 sketch - I guess you are not using the

#include <ESP8266WebServer.h>          // for the webserver

As mentioned - you should use this HTTP Server!
When you post your current code I will have a look at on the weekend.

13

Simply don’t use such a crude solution for your web page. Refer instead to what is known as the AJAX technique (Asynchronous JavaScript and XML).

This is a simple sketch that demonstrates the underlying principle.
If you want to see what happens when you click the buttons, open your browser's developer tools (typically by pressing F12) and go to the "Network" tab.

#include <ESP8266WiFi.h>
#include <ESP8266WebServer.h>

const char* ssid = "YOUR_SSID";
const char* password = "YOUR_PASSWORD";

ESP8266WebServer server(80);
const int ledPin = LED_BUILTIN; 
const char MAIN_page[] PROGMEM = R"rawliteral(
<!DOCTYPE html>
<html>
<head>
  <title>LED Control</title>
  <meta charset="UTF-8">
  <script>
    function sendCommand(cmd) {
      var xhr = new XMLHttpRequest();
      xhr.open("GET", "/led?cmd=" + cmd, true);
      xhr.onreadystatechange = function() {
        if (xhr.readyState === 4 && xhr.status === 200) {
          document.getElementById("status").innerText = xhr.responseText;
        }
      };
      xhr.send();
    }
  </script>
</head>
<body>
  <h1>ESP8266 LED Control</h1>
  <button onclick="sendCommand('on')">LED ON</button>
  <button onclick="sendCommand('off')">LED OFF</button>
  <p>Status: <span id="status">Unknown</span></p>
</body>
</html>
)rawliteral";

void handleRoot() {
  server.send_P(200, "text/html", MAIN_page);
}

void handleLED() {
  String cmd = server.arg("cmd");
  if (cmd == "on") {
    digitalWrite(ledPin, LOW); // LOW = led on with ESP8266 boards
    server.send(200, "text/plain", "LED is ON");
  } else if (cmd == "off") {
    digitalWrite(ledPin, HIGH); // HIGH = led off
    server.send(200, "text/plain", "LED is OFF");
  } else {
    server.send(400, "text/plain", "Invalid command");
  }
}

void setup() {
  Serial.begin(115200);
  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, HIGH); 

  WiFi.begin(ssid, password);
  Serial.print("Connecting to WiFi");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("\nConnected! IP: " + WiFi.localIP().toString());

  server.on("/", handleRoot);
  server.on("/led", handleLED);
  server.begin();
  Serial.println("HTTP server started");
}

void loop() {
  server.handleClient();
}
14

noiasca,
I haven't posted my code because it's way too long with multiple tabs and, more importantly, it's a mess. I'd be tared and feathered if I dared to post it here.
I have very little experience or training in today's modern languages and protocols. So I tend to copy and paste pieces of code from wherever I can find applicable logic.

cotestatnt's code uses <ESP8266WebServer.h> as you suggest and I am studying it and trying to learn.

Thanks,
Frank

15

cotestatnt, thank you

I compiled and tested your code and it works nicely. additionally, I learned a little about using the browser development tools.

I can barely hack my way through HTML code, javascript and XML are way over my head. Or perhaps better described as I don't have the time to study them (literally, I'm age 83).

So, hopefully as time permits, I'll be able to use pieces of your code in the future.
Thanks again
Frank

16

@frank2644

I have several webserver examples on my page for the ESP8266 also.

On this page you will find several examples from simple to more advanced technics:

The example 52 is such a simplified "toggle pins".
Basically it sends the form to an endpoint with handles commands and that endpoint advises the browser to do the redirect. These are just two lines of code at the end of the function.

webserver52
webserver52793×348 15.1 KB

Don't get confused with the several tabs, I just have put each "example" in a separate tab.

There's also an example how to update values on the page without reloading the whole page by using the newer FetchAPI.

... still not enough ...

P.S.:
If you would like to have a more fancy button, you could check my generic webserver.

grafik
grafik516×540 11.2 KB

it's the third link on my page.

17

Feel free to ask for further explanations if you want to go deeper, even regarding the HTML/JavaScript part.

18

If you're going to open the can of worms that is JavaScript, consider updating your examples to spare future generations from contemplating cryptic boilerplate like "xhr.readyState === 4": use the Fetch API

    async function sendCommand(cmd) {
      const response = await fetch("/led?cmd=" + cmd);
      if (response.ok) {
        document.getElementById("status").innerText = await response.text();
      }
    }

whose name also avoids the scary implication that XML is involved.

1 Like
19

Personally, I always use the Fetch API too.
The example in question was written in 10 seconds by an AI.
I had considered having it modified to use fetch, but since most examples and tutorials online still unfortunately use XMLHttpRequest(), I decided it was good enough as is.

Frankly, I don't understand why there's so much hostility/fear toward JavaScript in the Arduino world, and why some people keep proposing approaches based on examples that were already outdated 10 years ago - like building HTML response on runtime concatenating an huge String variable or pages that refresh the whole content or similar atrocities.

I'm not talking about your examples of course.

20

After some more thought I think I found a solution to my problem.

First let me explain that I have a bunch of projects that have the same problem. And I believe the solution presented above is to use the POST method rather than the GET method. I suspect this requires a significant rewrite of my existing sketches as it requires using the ESP8266WebServer/HTTPWebServer that I think uses a different command set.

Anyway, I made a slight modification to my above alternating button keyword solution. That is, instead of alternating, append a numeric suffix to the button parameters upon every button activation. i.e., String="led=/on" + millis();. This makes every subsequent button activation have a unique identifier and eliminates accidental activation for example when the back button or refresh occurs.

Hopefully my above explanation is not to convoluted.

I do see a minor complication if multiple client browsers/users are involved. Because a button activation by one user would invalidate the button keyword for other users. That simply means a button activation may not work the first time, but would work the second time. Not a terrible problem for my applications. BTW all my projects generally actually have only one user (me), but I do use a computer or a cell phone so effectively like two users so I might encounter this minor issue rarely.

If anyone sees any gotchas with my new solution please advise.

Thanks
Frank