MAC address discovery with ESP NOW connections

Projects Programming
1

I written a few ESP NOW apps to get the hang of the protocol which I think I am; but I cannot get away from the fact that I having to hardcode the MAC addresses of the ESP32s [Seeed ESP32C3 and ESP32C6] into my code.

Although in the real world this feels like a chicken and egg problem. One peer cannot send a message to the other before he has their MAC address, and vice versa.

But it isn't practical to do this in the real world? Is there some sort Bonjour protocol hiding in the implementation I haven't found yet? A discovery protocol that would let one peer know who his/her other peers were?

2

ESP-Now supports broadcast, multicast, and unicast. Each sender has a peer list for up to just over a dozen MAC addresses. Each recipient has to be added to the list to send there. You can also send to everyone on the list.

To broadcast, the special all-0xFF is used. You add that as a peer, and then send to it. Then everyone in range may receive it. There are various factors that determine whether a receiver gets any particular message. So you would probably broadcast periodically for a while.

The receiver always gets the sender's MAC in the callback. So now the receiver knows there is someone out there with that MAC, which "completes the circle", and in theory they can talk to each other.

And while it is unlikely to get a unicast message incorrectly sent to you, a receiver is likely to receive broadcasts from random senders nearby. That means you need some way of verifying the payload, and not to blindly act upon it. At a minimum by checking the length and format; and most securely by using cryptographic message signatures.

3

In my understanding, the MAC address should be hard coded, or stored in PROM to be compliant, further, MAC addresses are UNIQUE, they aren’t something you make up on a whim - e.g. DE AD BE EF, is a valid but nonsense address that exists on probably millions of hosts globally.

e.g.
“ The first three sets of two hexadecimal numbers in a MAC Address identify the card manufacturer , and this number is called OUI (organizationally unique identifier). It is always the same for NICs manufactured by the same company.”

Non-adherence to this protocol can lead to catastrophic network issues when arbitrary MACs are used on a WAN or the internet.

4

@wizard1066 I approach it this way

I collect the MAC of every device I intend to use plus a couple of extra in case I have a device failure. I add all these MAC addresses to an array on the master station.

Every remote station has a copy of the master MAC. Lets say I have 6 remotes but only intend to use 4.

The master will only communicate with a MAC that has been added to the peer list, the peer list is kept on the master. All the remotes have already been programmed with the master MAC

So I have 4 remotes all transmitting their MAC plus a message, each time the master receives a MAC I look to see if it is a member of the peer list, if it is not a member of the peer list I loop through the mac array and check to see if this MAC is an item of my mac array, if it is I add it to the peer list and break from the loop.

This is repeated 4 times at which point I know my network is complete and I stop looking for peers to add, application communication can then begin.

If a remote should fail or need some small modification it can be removed and replaced with one of the spares programmed earlier, the master will recognize it as a "known" mac and add it to the peer list seamlessly from the mac array.

5

???

6

I should have said, are expected to be unique…

If you look at many examples for add on NICs, they frequently use that nonsense MAC, which is likely unique on your LAN, for the first device, but will cause route issues if you add more.

7

have you considered using broadcast UDP messages that don't require knowing the addressess of recipients?

yes, the drawback is communication is unreliable, but periodic retransmission is one approach to improve reliability

i'm using this approach on a model RR signal system. Node broadcast block occupancy changes without knowing the # of other nodes. This also supports a display board that indicates occupied blocks. yse, the Wifi environment is relatively quiet

i'll guess that greater reliability can also be achieved by nodes recognizing node addresses from messages and build a table that can be used for recognizing ACKs and repeatedly sending msgs until ACKs are received from all recipients in a table

8

I like the UDP idea, but does Arduino IDE give you that level of resolution? Can you share some code that I could try?

On a related note I did find a solution of sorts that looks promising here on random nerd tutorials. He sets a soft AP within which he starts to listen for ESPNOW messages. On the client side, they connect and use the MAC broadcast address, you know, the FF:FF:FF, etc., to find out who else is running/listening on ESPNOW. They then exchange Mac Addresses across ESPNOW and start talking to the peeps who reply. That's what I think the gist of it is, anyway.

All that said, it's down to cook your own. I might try something similar that uses socket X to share Mac addresses. I run a softAP on 192.168.1.4, a server on which I also run a socket X that returns my IP address when someone connects to it, like the UNIX finger service. On the client, I watch for the event and get an IP address, and on doing so, I connect to socket X and get the Mac address of the ESP-NOW server. The server detects that connection and runs a socket X connection to the connecting terminal. Voilà you exchanged macs. As I think about it I don't need the softAP, I just setup the socket X and network events.

9

here's the wifi code from my model RR node
look at _wifiInit(), wifiSend() and _wifiCfgUdp() where _wifiMsg() is specified as a callback

functions prefixed with '_' would be private if this were a class


#ifdef ESP32
# include <WiFi.h>
#elif defined(ESP8266)
# include <ESP8266WiFi.h>
#endif

#include "AsyncUDP.h"

#include "eeprom.h"
#include "node.h"
#include "signals.h"
#include "wifi.h"

int dbgWifi = 1;

static void _wifiScan (void);
static void _wifiMsg  (const char *msg);

// -----------------------------------------------------------------------------
// WiFiClient          wifi;
AsyncUDP udp;

char host [STR_SIZE] = "";
char ssid [STR_SIZE] = "";
char pass [STR_SIZE] = "";

static int  port  = 4445;

// ---------------------------------------------------------
enum { ST_NUL, ST_INIT, ST_CHK, ST_CFG_UDP, ST_UP, ST_ERROR };

const char *wifiStStr [] = {
    "ST_NUL",
    "ST_INIT",
    "ST_CHK",
    "ST_CFG_UDP",
    "ST_UP",
    "ST_ERROR"
};

int state    = ST_NUL;
int stateLst = ST_NUL;

// -------------------------------------
static bool
_wifiCheck (void)
{
    static int           fails = 0;
    static unsigned long msecLst = 0;

    if ( (msec - msecLst) < 1000)
        return false;
    msecLst = msec;

    printf (" %s:", __func__);

#if 0
    if (6 <= fails)  {
        _wifiScan ();
        return false;
    }
#endif

    if (WL_CONNECTED != WiFi.status ())  {
        printf (" not connected\n");
        fails++;
        return false;
   }

   IPAddress ip = WiFi.localIP ();

   printf (" connected %d:%d:%d:%d\n", ip [0], ip[1], ip [2], ip[3]);

   return true;
}

// -------------------------------------
// https://arduino.clanweb.eu/udp-control-esp32.php?lang=en

static void
_wifiCfgUdp (void)
{
    printf (" %s:\n", __func__);

    if (udp.listen (port)) {
        Serial.print ("  UDP Listening on IP: ");
        Serial.println (WiFi.localIP ());

        udp.onPacket ([] (AsyncUDPPacket packet) {
#if 0
            Serial.print   ("UDP Packet Type: ");
            Serial.println (packet.isBroadcast ()
                        ? "Broadcast"
                        : packet.isMulticast () ? "Multicast" : "Unicast" );
            Serial.print   (" From: ");
            Serial.print   (packet.remoteIP ());
            Serial.print   (":");
            Serial.println (packet.remotePort ());
            Serial.print   (" To: ");
            Serial.print   (packet.localIP ());
            Serial.print   (":");
            Serial.println (packet.localPort ());
            Serial.print   (" Length: ");
            Serial.println (packet.length ());
            Serial.print   (" Data: ");
            Serial.write   (packet.data (), packet.length ());
            Serial.println ();
#endif

            _wifiMsg ((const char *) packet.data ());
        });
  }
}

// -------------------------------------
// connect to wifi
void _wifiInit (void)
{
    printf (" %s:\n", __func__);

    _wifiScan ();
    WiFi.mode (WIFI_STA);

    WiFi.hostname (host);

    printf ("%s: ssid %s, pass %s\n", __func__, ssid, pass);
    WiFi.begin (ssid, pass);
}

// -------------------------------------
static void
_wifiMsg (
    const char *msg)
{
    printf (" %s:\n", __func__);

    sigMsg (msg);
}

// -------------------------------------
void
wifiReset (void)
{
    printf ("%s: ssid %s, pass %s, host %s\n", __func__, ssid, pass, host);
    state = ST_NUL;
}

// -------------------------------------
// https://openlabpro.com/guide/scanning-of-wifi-on-esp32-controller/
static void
_wifiScan (void)
{
    printf ("%s:\n", __func__);

    // WiFi.scanNetworks will return the number of networks found
    WiFi.mode (WIFI_OFF);
    int nNet = WiFi.scanNetworks ();
    if (nNet == 0) {
        printf ("  %s: no networks found\n", __func__);
    }
    else {
        printf ("  %s: %2d networks found\n", __func__, nNet);
        if (0 > nNet)
            nNet = -nNet;
        for (int i = 0; i < nNet; ++i) {
            printf (" %s: %2d - %s (%d) %s\n", __func__, i,
                WiFi.SSID (i).c_str (),
                WiFi.RSSI (i),
                WiFi.encryptionType (i) == WIFI_AUTH_OPEN ? "open" : "locked");
        }
    }
}

// ---------------------------------------------------------
// common routine for sending strings to wifi and flushing
void
wifiSend (
    const char*  msg )
{
    if (ST_UP != state)
        return;

    if (dbgWifi)  {
        printf ("wifiSend: %s\n", msg);
    }

    udp.broadcast (msg);
}

// -------------------------------------
void
wifiMonitor (void)
{
    if (stateLst != state)
        printf ("%s: %d %s\n", __func__, state, wifiStStr [state]);
    stateLst = state;

    switch (state)  {
    case ST_NUL:
        if (ssid [0])
            state = ST_INIT;
        else {
            printf ("%s: no SSID\n", __func__);
            state = ST_ERROR;
        }
        break;

    case ST_INIT:
        _wifiInit ();
        state = ST_CHK;
        break;

    case ST_CHK:
        if (_wifiCheck ())
            state = ST_CFG_UDP;
        break;

    case ST_CFG_UDP:
        _wifiCfgUdp ();
        state = ST_UP;
        break;

    case ST_UP:
        break;

    case ST_ERROR:
    default:
        break;
    }
}
10

I hit a tiny snag here that I think is normal? If I start in WIFI_AP mode I get a slightly different MAC address to in WIFI_STA. Here is the code if you want to try it. The difference is consistent. So in I get 64:E8:33:81:5E:C9 or 64:E8:33:81:5E:C8.

#include "WiFi.h"
#include <WiFiClient.h>

WiFiServer server(49156);
WiFiClient client1;

// pure data SSID and No password
const char *ssid2 = "pdSSID";
const char *password2 = NULL;

void setup() {
  Serial.begin(115200);
  while (!Serial);
  //initAP();
  initWiFi();

  Serial.println("Setup done");
  server.begin();
  Serial.println("Running server");
}

const char* ssid = "domain";
const char* password = "password";

void initWiFi() {
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.print("Connecting to WiFi ..");
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print('.');
    delay(1000);
  }
  Serial.println();
  Serial.println(WiFi.localIP());
  Serial.println(WiFi.macAddress());
}

void initAP() {
  WiFi.mode(WIFI_AP);
  WiFi.softAP(ssid2, password2);
  Serial.print("Connecting 2 WiFi ..");
  Serial.print("AP IP Address: ");

  Serial.println(WiFi.macAddress());
  Serial.println(WiFi.softAPIP());
}

void loop() {
  client1 = server.available();   // listen for incoming clients
  if (client1) {                             // if you get a client,
    Serial.println("New Client.");           // print a message out the serial port
    Serial.println(WiFi.softAPmacAddress());
    client1.println(WiFi.softAPmacAddress());
    delay(500);
    client1.stop();
  }
  delay(500);
}

You can use telnet 49156 to connect to and get answers from a terminal window. Obviously, if you're in AP mode, you'll need to join the pdSSID domain else telnet will not work.

11

Greg,
Slept on your proposal and looked up the library this morning, and put together this simple piece of code. Works like a charm. I think I'll use this to setup ESP-NOW! Thanks

//UDP Client
#include "WiFi.h"
#include "AsyncUDP.h"

const char* ssid = "uzr";
const char* password = "";

AsyncUDP udp;
int count = 0;

void setup() {
  Serial.begin(115200);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  if (WiFi.waitForConnectResult() != WL_CONNECTED) {
    Serial.println("WiFi Failed");
    while (1) {
      delay(1000);
    }
  }
}

void loop() {
  delay(24000);
  //Send broadcast
  Serial.println("broadcasting...");
  udp.broadcastTo("Anyone here?",1234);
}

//UDP Server
#include "WiFi.h"
#include "AsyncUDP.h"

const char* ssid = "uzr";
const char* password = "";

AsyncUDP udp;
int count = 0;

void setup() {
  Serial.begin(115200);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  if (WiFi.waitForConnectResult() != WL_CONNECTED) {
    Serial.println("WiFi Failed");
    while (1) {
      delay(1000);
    }
  }
  
  if (udp.listen(1234)) {
    Serial.print("UDP Listening on IP: ");
    Serial.println(WiFi.localIP());
    udp.onPacket([](AsyncUDPPacket packet) {
      if (count < 8) {
        Serial.print("UDP Packet Type: ");
        Serial.print(packet.isBroadcast() ? "Broadcast" : packet.isMulticast() ? "Multicast" : "Unicast");
        Serial.print(", From: ");
        Serial.print(packet.remoteIP());
        Serial.print(":");
        Serial.print(packet.remotePort());
        Serial.print(", To: ");
        Serial.print(packet.localIP());
        Serial.print(":");
        Serial.print(packet.localPort());
        Serial.print(", Length: ");
        Serial.print(packet.length());
        Serial.print(", Data: ");
        Serial.write(packet.data(), packet.length());
        Serial.println();
        //reply to the client
        packet.printf("Got %u bytes of data", packet.length());
      count++;
      }
    });
  }
}

void loop() {
  delay(1000);
}

The client send a packet out to all stations on its local network searching for port 1234. The server in the meantime is listening to port 1234. You can run a server and send out a broadcast at the same time.

Simple, clean, elegant... what can say, Greg, your the man of the hour!

12

I’ll admit up front that I haven’t tried this, but …. wondering if a hybrid approach might work. On start up the Master would send Discovery packets via Broadcast at some periodic rate. Upon receiving a discovery packet, each node would wait a random amount of time (to minimize collisions) and then send a Acknowledge packet back via Unicast to the Master with its own Address (which would be part of the packet anyway) and its function in the layout. Upon receiving the Acknowledge packet, the Master would send it back its own Acknowledge packet via Unicast that would tell the slave node to stop replying to the Discovery packets. The process then continues until no replies are received (timeout period) to theBroadcast Discovery packets. Now the Master knows the address and function of every Slave node.

13

what problem (there are problems) does this solve?

14

Just a different way of doing it. I like that once discovery is complete, all operational communications can then be via Unicast. A preference.

15

guess it's not clear what the requirement is

presumably for a unicast application, each node is aware of the other nodes by address. no need to discover

at least for my application, there's a need to share information with every other node ... without needing to know what they are. And it would be burdensome to discover N other nodes and then to have to send some msg N times.

16

There's a need to discover if the nodes' addresses are not known ahead of time and hardcoded. I believe that was the point of @wizard1066 original question. Discovery allows the master the find the node addresses dynamically, no need to hardcode them. And, no need to re-hardcode them if a node is replaced.

My proposal said nothing about node-to-node communications. It detailed a method for the Master to discover the address of every Slave node. After discovery, all Master <--> Slave communication can be via Unicast. I prefer that to Broadcast.

17

maybe the OP can comment. I'm confused how a node would know who to send a msg to without knowing the address/node-name. isn't this kinda a chicken and egg problem.

how does discovering a address tell you who to send a msg to, assuming there are multiple nodes and each is only suppose to receive specific information, unless each node describes what they want to know. At that point is MQTT a better approach

18

I was describing a topology with one Master node and multiple Slave nodes. So, as I said:

Thus, after receiving all acknowledgements, the Master knows the address of all Slave nodes and their Function ID. Each Slave node also knows the address of the Master. That's all you need for Master <--> Slave Unicom traffic. The Master does it's "mastering" by Function ID ... "I want the node with Function ID 'abc' to perform the action 'xyz'. So I look up the address for the node with Function ID 'abc' and send it a Unicom message telling it to perform action 'xyz'."

Also, if the application doesn't require high streaming speed (as I imagine an model train layout controller does not), then I'd use TCP vs UDP messaging.

But, it appears that I misread OP's first post anyway as peer-to-peer comms is mentioned. So, the Master / Slave model does not hold.

19

still an interesting approach to do such things

20

ESP-NOW auto discovery is relatively easy, and you don't need to use wifi functions at all.
There are 2 types of auto discovery; triggered by a 'pair' button on devices, or run at every start up. I'll start with a simple 2 devices by pair button explaination -
// All devices are hard code paired with 'broadcast'.
// On either device hold the pair button during boot up.
// It will flash an led and check for a flag set by OnDataRecv() over & over for up to 5s.
// If a broadcast message arrives it will have the initiating device's mac,
// so pair with it, then send this responder device's mac to it. Paired this end.
// If no ESP-NOW broadcast comes in after 5s it will assume it is to initiate pairing,
// so will flash the led faster while broadcasting its mac ID out for 10s.
// For the second device watch the led on the first to be booted.
// When the first device starts fast flashing, hold the pair button on the
// second device while powering On, then release.
// When the new responder replies it will provide it's mac.
// Pair to it. Paired this end.
// Flashing stops once paired. No led = success, led on = failed
// If sucessfull, both devices store the mac in EEPROM.

For true auto discovery the sequence gets run at every start up with each device testing for incomming broadcasts after a random delay and if none come in it initiates broadcasting.
And repeats broadcasting after finding each peer until a timeout. This works when all devices get powered On at the same time. For random start times a little more code is required, but it is still pretty simple.

After pairing with all devices the initiator then sends a system list to all peers. Each peer then checks its previously stored peer list against this and ... does what the programmer decides to do.

All the mac data is in the header of each packet exchange. To access it look at the struct esp_now_recv_info at (editor split URL, rejoin to use) -

https://docs.espressif.com/projects/esp-idf/en/latest/esp32c5/api-reference/network/esp_now.html

Don't forget that the active pair list is limited to 20 devices, or 6 if encrypted. (But you can swap inactive to/from the list at will).
Also ensure you are using the current V2 of the library, it is much better than V1.

New info - Expressif has an init and discover all example !

bye.