Arduino Nano Web Server & Ajax (not quite) working

I am using a Nano with ENC28J60. My idea is to establish a Web Server that will periodically refresh some data on the webpage without reloading the whole page. I read about Ajax and decided to give it a go. Found a pretty straightforward example here:

There are several other projects online that use practically the same code, only with different sensors etc. So it should be working. I am using the example from the link above, only changed the IP address and port according to my router settings. And of course, using the library EthernetENC.h. Here is what happens:

  • If I reset the Arduino while there is a browser page open with the Arduino IP address in the address bar, the program starts to run as it should, showing the latest switch status. However, sometimes it stops after a while and produces the opening text:

Arduino AJAX Switch Status
Switch state: Not requested...

  • If Arduino first starts to run and only after that I open the page, the program doesn't work. I just get the static text:

Arduino AJAX Switch Status
Arduino AJAX Switch Status
Switch state: Not requested...

And you saw it right. The first line is repeated twice, although it is only stated once in the code. I've been thinking about it, tinkering with the code, searching online for a few days, but no success. Any help?

That is some old school Ajax (I might say "even for 2013", but my memory is fuzzy)

You've got a web server that sends either

  • an HTML page with AJAX
    • the page body contains an <h1>, and a <p> to display the dynamic content
  • some text to insert in that <p>

If the server instead sends the page in response to the AJAX request, after it is inserted, then you get the heading twice.

The server decides what to send with

indexOf > -1 is equivalent to "contains" or "includes", and is frankly a pretty sloppy way of checking, since it will scan the entire string; it both (a) wastes time when it is not there, and (b) may find text in other than the intended place. Anyway, there's already a statement to print that variable whenever the server sends a response; what does it say?

The web page makes the request with JavaScript

Even though nocache starts with a & instead of what is surely intended to be a ?, the indexOf test should work either way.

@kenb4, from your post it seems that there are no errors in the code?

Here is what it says in the serial monitor:

GET /ajax_switch&nocache=6340.785373073477 HTTP/1.1
Host: 192.168.1.71:6501
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
Accept: */*
Referer: http://192.168.1.71:6501/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,it;q=0.8,hr;q=0.7

And this when I click refresh on the webpage:

GET / HTTP/1.1
Host: 192.168.1.71:6501
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, dGET /ajax_switch&nocache=751677.66259914 HTTP/1.1
Host: 192.168.1.71:6501
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
Accept: */*
Referer: http://192.168.1.71:6501/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,it;q=0.8,hr;q=0.7

GET /favicon.ico HTTP/1.1
Host: 192.168.1.71:6501
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
Accept: image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8
Referer: http://192.168.1.71:6501/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,it;q=0.8,hr;q=0.7

I see two GET requests after page refresh. Could this be part of the problem?

Is that two or three requests? If you look in the middle of the first one, it says

The Accept-Encoding header is supposed to be gzip, deflate, as it says in two other places, but here it is cut off, and another request starts with GET /ajax_switch So is that a copy/paste error, or is that what is actually seen as a single request?

The /favicon.ico request will get the HTML page as a response, since it does not pass the indexOf("ajax_switch") > -1 test. That is not a valid response for this request for an icon, and the browser will likely ignore it, and keep trying. It's something that could be improved, but likely separate from your main problem.

If you use the browser's developer tools, and look at the Network monitor, you should see separate requests for the main page and the ajax_switch XHR call, and the response for each. If you're not getting the appropriate content, you can add some debugging code to verify why it's sending the HTML page versus the XHR text.

Thanks for your interest in this topic. Here is my code:

#include <SPI.h>
#include <EthernetENC.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0x47 };
IPAddress ip(192, 168, 1, 71);
EthernetServer server(6501);

String HTTP_req;

int i = 0;

void setup()
{
  Ethernet.begin(mac, ip);
  server.begin();
  Serial.begin(9600);
}


void loop()
{
  EthernetClient client = server.available();

  if (client)
  {
    boolean currentLineIsBlank = true;
    while (client.connected())
    {
      if (client.available())
      {
        char c = client.read();
        HTTP_req += c;
        if (c == '\n' && currentLineIsBlank)
        {
          client.print(F("HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nConnection: keep-alive\r\n\r\n"));

          if (HTTP_req.indexOf("ajax_switch") > -1)
          {
            GetSwitchState(client);
          }
          else
          {
            client.print(F("<!DOCTYPE html><html><head><title>Arduino Web Page</title <script>"));

            client.println("function GetSwitchState() {");
            client.println("nocache = \"&nocache=\"\+ Math.random() * 1000000;");
            client.println("var request = new XMLHttpRequest();");
            client.println("request.onreadystatechange = function() {");
            client.println("if (this.readyState == 4) {");
            client.println("if (this.status == 200) {");
            client.println("if (this.responseText != null) {");
            client.println("document.getElementById(\"switch_txt\")\.innerHTML = this.responseText;");
            client.println("}}}}");
            client.println("request.open(\"GET\", \"ajax_switch\" + nocache, true);");
            client.println("request.send(null);");
            client.println("setTimeout('GetSwitchState()', 2000);");
            client.println("}");
          
            client.print(F("</script></head><body onload=\"GetSwitchState()\"><h1>Arduino AJAX Switch Status</h1><div id=\"switch_txt\">Starting...</div></body></html>"));
          }
              
          Serial.print(HTTP_req);
          HTTP_req = "";
          break;
        }
          
        if (c == '\n')
        {
          currentLineIsBlank = true;
        } 
        else if (c != '\r')
        {
          currentLineIsBlank = false;
        }
      }
    }
      
    delay(1);
    client.stop();
  }
}


void GetSwitchState(EthernetClient cl)
{
  i++;
  cl.println(i);
}

On each Ajax request the value of "i" should increase by 1 and be refreshed on the webpage. Now I will repeat myself a bit, but hopefully things will be clearer. Like I wrote before, this works after reset. On reset the page is loaded and then Ajax request are issued in regular intervals and the value of "i" is updated and displayed.. Again, the Ajax request looks like this:

GET /ajax_switch&nocache=724877.1695980653 HTTP/1.1
Host: 192.168.1.71:6501
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
Accept: */ *
Referer: http://192.168.1.71:6501/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,it;q=0.8,hr;q=0.7

Now, here is what happens when I refresh the page in the browser. From the serial monitor:

GET / HTTP/1.1
Host: 192.168.1.71:6501
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/ *;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate
Accept-LanGET /ajax_switch&nocache=645568.7831047487 HTTP/1.1
Host: 192.168.1.71:6501
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
Accept: */ *
Referer: http://192.168.1.71:6501/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,it;q=0.8,hr;q=0.7

GET /favicon.ico HTTP/1.1
Host: 192.168.1.71:6501
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
Accept: image/avif,image/webp,image/apng,image/svg+xml,image/ *,*/ *;q=0.8
Referer: http://192.168.1.71:6501/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,it;q=0.8,hr;q=0.7

Note: I inserted some spaces between / and * so they don't look like comments when formatted. Anyway, what you asked before is not a copy-paste error. The request from the browser on refresh is somehow "interrupted" by an Ajax request. It seems to jump right in the middle of it:

...
Accept-Encoding: gzip, deflate
Accept-LanGET /ajax_switch&nocache=645568.7831047487 HTTP/1.1
...

I've been trying to rearrange the code, adding some if-else conditions to differentiate between browser and Ajax requests, but to no avail. The result is always this:

Developer tools:

Indeed, I get the large text twice and the dynamic text doesn't work, it's stuck at the initial text. It seems that the Ajax request that somehow jumps into the browser request on refresh is a problem. I even chose a rather long time between Ajax requests (5 seconds) to make sure that I refresh between them, not at the same time.

What also puzzles me is that I see no difference in the code between me and several, apparently successful, examples online.

There's at least one consequential typo in the code you posted, so try this segment instead:

        if (c == '\n' && currentLineIsBlank) {
          client.print(F("HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nConnection: close\r\n\r\n"));

          if (HTTP_req.startsWith("GET /ajax_switch")) {
            GetSwitchState(client);
          } else {
            client.print(F(R"~~~(<!DOCTYPE html>
<html>
<head>
<title>Arduino Web Page</title>
<script>
function GetSwitchState() {
  var nocache = "nocache=" + Math.random() * 1000000;
  var request = new XMLHttpRequest();
  request.onreadystatechange = function() {
    if (this.readyState == 4) {
      if (this.status == 200) {
        if (this.responseText != null) {
          document.getElementById("switch_txt").innerHTML = this.responseText;
        }}}}
  request.open("GET", "ajax_switch?" + nocache, true);
  request.send(null);
  setTimeout(GetSwitchState, 1000);
}
</script>
</head>
<body onload="GetSwitchState()">
<h1>Arduino AJAX Switch Status</h1>
<div id="switch_txt">Starting...</div>
</body>
</html>
)~~~"));
          }

          Serial.println("<<\n>>");
          Serial.print(HTTP_req);
          HTTP_req = "";
          break;
        }
  • The connection is closed after the response, so don't lie and say Connection: keep-alive -- it is Connection: close
  • Instead of the flaky indexOf check, use the more accurate startsWith along with the request method, GET
    • this would theoretically fix the opposite of what you're seeing
  • Use a raw string for the HTML page content
    • The F() macro can be applied in a single shot
    • Code is less noisy
    • For HTML in particular, no need to escape double-quotes
  • You had </title <script>, which breaks the HTML
  • Added an extra << >> separator when printing the requests to see where they end and begin

Thanks a lot! It seems to be working fine after all these changes. And I learned a few things too. Here is the code:

#include <SPI.h>
#include <EthernetENC.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0x47 };
IPAddress ip(192, 168, 1, 71);
EthernetServer server(6501);

String HTTP_req;

int i = 0;

void setup()
{
  Ethernet.begin(mac, ip);
  server.begin();
  Serial.begin(9600);
}


void loop()
{
  EthernetClient client = server.available();

  if (client)
  {
    boolean currentLineIsBlank = true;
    while (client.connected())
    {
      if (client.available())
      {
        char c = client.read();
        HTTP_req += c;
        if (c == '\n' && currentLineIsBlank)
        {
          client.print(F("HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nConnection: close\r\n\r\n"));
          if (HTTP_req.startsWith("GET /ajax_switch"))
          {
            GetSwitchState(client);
          }
          else
          {
            client.print(F(R"~~~(
              <!DOCTYPE html>
              <html>
              <head>
              <title>Arduino Web Page</title>
              <script>

              function GetSwitchState()
              {
                var nocache = "nocache=" + Math.random() * 1000000;
                var request = new XMLHttpRequest();
                request.onreadystatechange = function()
                {
                  if (this.readyState == 4)
                  {
                    if (this.status == 200)
                    {
                      if (this.responseText != null)
                      {
                        document.getElementById("switch_txt").innerHTML = this.responseText;
                      }
                    }
                  }
                }
                request.open("GET", "ajax_switch?" + nocache, true);
                request.send(null);
                setTimeout(GetSwitchState, 1000);
              }
              </script>
              </head>
              <body onload="GetSwitchState()">
              <h1>Arduino AJAX Switch Status</h1>
              <div id="switch_txt">Starting...</div>
              </body>
              </html>
              )~~~"));
          }
          Serial.println("<<\n>>");
          Serial.print(HTTP_req);
          HTTP_req = "";
          break;
        }
          
        if (c == '\n')
        {
          currentLineIsBlank = true;
        } 
        else if (c != '\r')
        {
          currentLineIsBlank = false;
        }
      }
    }
    delay(1);
    client.stop();
  }
}


void GetSwitchState(EthernetClient cl)
{
  i++;
  cl.println(i);
}