Avoid the user to type the IP address of Arduino webserver

I'm working on an Arduino project. It is a simple device with WiFi connection, but it is not IoT - that means I have no Cloud or external hosting to rely on.

Using a physical button, the user can trigger the provisioning sequence: the device will turn on an hotspot and when the user connects it redirects to a landing page (using a captive portal) where he can select his own WiFi network.

After that the device will connect to the user's WiFi network. So far so good.
Now, to reach the HTML page of the Arduino webserver the user has to type in his browser the (dynamic) IP address of the device. This is shown on a display.

The boss is concerned about this step, since he thinks most people are not able to do and he asked for a different way to connect to the internal webserver without the needs of typing the IP address.

Some ideas I've considered and discarded:

  • QRCode on the display: it would work, but the display is too small (and we cannot change it now)

  • hostname: it would be "simpler" since the user would type a word instead of a number ( :thinking:) but it does not always work because it depends on the router's configuration

  • the device, after getting the IP, can activate again the hotspot. The user can connect to it for the second time and a page will show a link to that IP address. Clicking on it the browser will fail (because we're not connected to the user's network) but he can bookmark the page for later use. I think is cumbersome...

Any other idea?

Hi @Mark81. The common solution is to use a "captive portal". This way the user only needs to connect to the access point of the device and open their web browser. The web browser will then automatically open the provisioning page.

This is used by the popular "WiFiManager" library, as well as similar alternative libraries.

That's exactly what I wrote in my post but this works only for the provisioning stage.
After the provisioning has ended and the device connects to the home network, of course I cannot use the captive portal anymore.

So the user needs to know and type the IP address in order to reach the pages of the webserver.

Ah I see now. Sorry about that.

You didn't specify which microcontroller you're working with, but with ESP32 there are some examples where a mDNS (multicast DNS) is used to reach the device using the local address in this form:

http://hostname.local

I assume there are similar possibilities with other frameworks too

1 Like

It is an ESP32.

Yes, the hostname solution was already in my question (second point). But it works only if the router is enabled for this service, so it cannot be a general solution because for some users wouldn't work.

Really? I never realized that...
I use it often, both with my home router and a smartphone as a hotspot, and it has always worked, so I've never encountered this issue.

What router setting could prevent mDNS from functioning?

Not sure what is the specific setting, but we encountered a lot of issues, and a search for "mDNS fail" leads to a lot of results. Some of the problems I remember:

  • as said, with some routers there was no way to reach the device using the hostname
  • in other scenarios it worked with some smartphones and not with others, but the same non-working smartphones were able to reach the device using a different router
  • some routers append a different domain: i.e. instead of hostname.local you must use hostname.lan, hostname.fritzbox and so on

There are so many variables, hence it's not a reliable solution for the end user.

1 Like

I have a method which basically forwards any URL coming from the user when it connects to the ESP as an access point. This could of course also be an empty URL. I set to a website alias, which does complicate things a bit because a phone will start to look through it's search engine to see if it can resolve it, so it may require 5G to be turned off.

Anyway, it sounds like that would do the trick for you, but i can not find or remember where i got it from It was a modification of DNSserver.h / .cpp

It required renaming it (i went for patchDNSserver.h /.cpp ) to make it includable.
the .h

#ifndef PatchDNSServer_h
#define PatchDNSServer_h
#include <WiFiUdp.h>

#define DNS_QR_QUERY 0
#define DNS_QR_RESPONSE 1
#define DNS_OPCODE_QUERY 0

enum class DNSReplyCode
{
  NoError = 0,
  FormError = 1,
  ServerFailure = 2,
  NonExistentDomain = 3,
  NotImplemented = 4,
  Refused = 5,
  YXDomain = 6,
  YXRRSet = 7,
  NXRRSet = 8
};

struct DNSHeader
{
  uint16_t ID;               // identification number
  unsigned char RD : 1;      // recursion desired
  unsigned char TC : 1;      // truncated message
  unsigned char AA : 1;      // authoritive answer
  unsigned char OPCode : 4;  // message_type
  unsigned char QR : 1;      // query/response flag
  unsigned char RCode : 4;   // response code
  unsigned char Z : 3;       // its z! reserved
  unsigned char RA : 1;      // recursion available
  uint16_t QDCount;          // number of question entries
  uint16_t ANCount;          // number of answer entries
  uint16_t NSCount;          // number of authority entries
  uint16_t ARCount;          // number of resource entries
};

class DNSServer
{
  public:
    DNSServer();
    void processNextRequest();
    void setErrorReplyCode(const DNSReplyCode &replyCode);
    void setTTL(const uint32_t &ttl);

    // Returns true if successful, false if there are no sockets available
    bool start(const uint16_t &port,
              const String &domainName,
              const IPAddress &resolvedIP);
    // stops the DNS server
    void stop();

  private:
    WiFiUDP _udp;
    uint16_t _port;
    String _domainName;
    unsigned char _resolvedIP[4];
    int _currentPacketSize;
    unsigned char* _buffer;
    DNSHeader* _dnsHeader;
    uint32_t _ttl;
    DNSReplyCode _errorReplyCode;

    void downcaseAndRemoveWwwPrefix(String &domainName);
    String getDomainNameWithoutWwwPrefix();
    bool requestIncludesOnlyOneQuestion();
    void replyWithIP();
    void replyWithCustomCode();
};
#endif

the .cpp

#include "PatchDNSServer.h"
#include <lwip/def.h>
#include <Arduino.h>

//#define DEBUG
//#define DEBUG_OUTPUT Serial

DNSServer::DNSServer()
{
  _ttl = htonl(60);
  _errorReplyCode = DNSReplyCode::NonExistentDomain;
}

bool DNSServer::start(const uint16_t &port, const String &domainName,
                     const IPAddress &resolvedIP)
{
  _port = port;
  _domainName = domainName;
  _resolvedIP[0] = resolvedIP[0];
  _resolvedIP[1] = resolvedIP[1];
  _resolvedIP[2] = resolvedIP[2];
  _resolvedIP[3] = resolvedIP[3];
  downcaseAndRemoveWwwPrefix(_domainName);
  return _udp.begin(_port) == 1;
}

void DNSServer::setErrorReplyCode(const DNSReplyCode &replyCode)
{
  _errorReplyCode = replyCode;
}

void DNSServer::setTTL(const uint32_t &ttl)
{
  _ttl = htonl(ttl);
}

void DNSServer::stop()
{
  _udp.stop();
}

void DNSServer::downcaseAndRemoveWwwPrefix(String &domainName)
{
  domainName.toLowerCase();
  domainName.replace("www.", "");
}

void DNSServer::processNextRequest()
{
  _currentPacketSize = _udp.parsePacket();
  if (_currentPacketSize)
  {
    _buffer = (unsigned char*)malloc(_currentPacketSize * sizeof(char));
    _udp.read(_buffer, _currentPacketSize);
    _dnsHeader = (DNSHeader*) _buffer;

    if (_dnsHeader->QR == DNS_QR_QUERY &&
        _dnsHeader->OPCode == DNS_OPCODE_QUERY &&
        requestIncludesOnlyOneQuestion() &&
        (_domainName == "*" || getDomainNameWithoutWwwPrefix() == _domainName)
       )
    {
      replyWithIP();
    }
    else if (_dnsHeader->QR == DNS_QR_QUERY)
    {
      replyWithCustomCode();
    }

    free(_buffer);
  }
}

bool DNSServer::requestIncludesOnlyOneQuestion()
{
  return ntohs(_dnsHeader->QDCount) == 1 &&
         _dnsHeader->ANCount == 0 &&
         _dnsHeader->NSCount == 0 &&
         _dnsHeader->ARCount == 0;
}

String DNSServer::getDomainNameWithoutWwwPrefix()
{
  String parsedDomainName = "";
  unsigned char *start = _buffer + 12;
  if (*start == 0)
  {
    return parsedDomainName;
  }
  int pos = 0;
  while(true)
  {
    unsigned char labelLength = *(start + pos);
    for(int i = 0; i < labelLength; i++)
    {
      pos++;
      parsedDomainName += (char)*(start + pos);
    }
    pos++;
    if (*(start + pos) == 0)
    {
      downcaseAndRemoveWwwPrefix(parsedDomainName);
      return parsedDomainName;
    }
    else
    {
      parsedDomainName += ".";
    }
  }
}

void DNSServer::replyWithIP()
{
  _dnsHeader->QR = DNS_QR_RESPONSE;
  _dnsHeader->ANCount = _dnsHeader->QDCount;
  _dnsHeader->QDCount = _dnsHeader->QDCount; 
  //_dnsHeader->RA = 1;  

  _udp.beginPacket(_udp.remoteIP(), _udp.remotePort());
  _udp.write(_buffer, _currentPacketSize);

  _udp.write((uint8_t)192); //  answer name is a pointer
  _udp.write((uint8_t)12);  // pointer to offset at 0x00c

  _udp.write((uint8_t)0);   // 0x0001  answer is type A query (host address)
  _udp.write((uint8_t)1);

  _udp.write((uint8_t)0);   //0x0001 answer is class IN (internet address)
  _udp.write((uint8_t)1);
 
  _udp.write((unsigned char*)&_ttl, 4);

  // Length of RData is 4 bytes (because, in this case, RData is IPv4)
  _udp.write((uint8_t)0);
  _udp.write((uint8_t)4);
  _udp.write(_resolvedIP, sizeof(_resolvedIP));
  _udp.endPacket();



  #ifdef DEBUG
    DEBUG_OUTPUT.print("DNS responds: ");
    DEBUG_OUTPUT.print(_resolvedIP[0]);
    DEBUG_OUTPUT.print(".");
    DEBUG_OUTPUT.print(_resolvedIP[1]);
    DEBUG_OUTPUT.print(".");
    DEBUG_OUTPUT.print(_resolvedIP[2]);
    DEBUG_OUTPUT.print(".");
    DEBUG_OUTPUT.print(_resolvedIP[3]);
    DEBUG_OUTPUT.print(" for ");
    DEBUG_OUTPUT.println(getDomainNameWithoutWwwPrefix());
  #endif
}

void DNSServer::replyWithCustomCode()
{
  _dnsHeader->QR = DNS_QR_RESPONSE;
  _dnsHeader->RCode = (unsigned char)_errorReplyCode;
  _dnsHeader->QDCount = 0;

  _udp.beginPacket(_udp.remoteIP(), _udp.remotePort());
  _udp.write(_buffer, sizeof(DNSHeader));
  _udp.endPacket();
}

And the implementation is easy enough.

#include <PatchDNSServer.h>

#define DNS_PORT 53

DNSServer dnsServer;

void setup() {
  dnsServer.start(DNS_PORT, "*", IPAddress(192, 168, 4, 1));
  // dnsServer.start(DNS_PORT, "blabla.com", IPAddress(192, 168, 4, 1));
}

void loop() {
  dnsServer.processNextRequest();
}

This works for connecting to it as an AP. Through the router it is a different matter. There may be newer ways to do the same, i think it was more or less a fix to part of a core that wasn't working properly.

Anyway, provided as is (compiles and works on all ESP cores i have used)

1 Like

Thanks for the hint! As far as I understand it is something similar to the captive portal.

Unfortunately, my needs it strictly related to the home router. I.e. when the ESP32 is in station mode and connected to the same network of the user's smartphone.

In that case mDNS is the only real solution, with it's reliability issues. I have some, though it tends to be related to a device connecting after my laptop has connected.
And of course there is the 'Android' issue, where Android (unlike IOS) still doesn't support mDNS lookup as far as i know.

1 Like

Called client isolation, a security feature. If this is enabled, you're basically screwed.

Since android 12(?) it does, but of course not all Android phones are that new. Apple devices have always relied on this type of technology ever since the 80's with the Appletalk. In the 00's it begun the transition from Appletalk to Bonjour a.k.a mDNS since TCP/IP is superior to Appletalk.

DHCP Option 15: Domain suffix to add when only a hostname is provided.

Some of these obstacles I can't see how they can be avoided, client isolation being the brick wall. Having the user to change settings in a router is a challenge in itself, more so if they don't have access.

start with mDNS.
Add mDNS SD
Write an IOS App and a Google App which discovers your device (based on SD in the network) and open the browser with the respective IP.
In short: provide an app to find the device.

If we were to write an IOS/Google app to discover the device, we would also rely on it to manage the options! The point of using the internal webserver is to provide a simple way to configure the device without developing (and maintain) apps.

But how will you find the webserver on your home network without the IP address unless there is something that tells you where it is ? The only way is either mDNS, or to log onto the device directly first.

You want to enter the house but the only one that knows where the house is, is behind the door of the house.

1 Like

That's exactly my question!

I thought by now it should be clear to you that what you are asking can normally be done with mDNS. You could on the other hand start by broadcasting UDP packages the same way ArtNet protocol does, but that requires an App to send these packages to which your ESP can respond or an App that receives broadcasted packages.

Maybe to that question the answer is just a simple NO !

I remember several IP cameras which came with a "device finder app" despite having a webconsole.

You (your boss) raised concerns regarding the capability of your customers to find the right IP address. A "device finder app" might be a solution (and is done by other manufacturers).

A possible solution could be to use a captive portal to implement a WiFi manager that allows the user to set the WiFi credentials and establish a connection with the specified SSID.

Once the IP address is obtained from the router, this information should be provided as feedback to the user.
If the ESP32 is left in mixed mode with WiFi.mode(WIFI_AP_STA), this is possible.
I use this exact approach in a library I developed to simplify the implementation of web servers with ESP microcontrollers.

This is the captive portal webpage, as indicated by the browser being connected to the default IP address: 192.168.4.1

And this is the feedback after successfull connection. The message is still incomplete; the address in the format http://hostname.local/ is missing. I'm still working on it.

1 Like

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.