Need help on esp32 w5000 mqtt with ssl

Hello, I use esp32, w5500 and sslclient lib, pubsub lib trying to connect to mqtt broker with ca.crt, client.crt and client.key. similar to test.mosquitto.org on port 8884

And no succeed yet. Would really appreciate any help.

Thank you

Try Google on " ESP... + mqtt".
Pretty sure this has been done successfully before, maybe using other libs, software.

so far, found one but no respond yet. currently trying other lib

Partly because PubSubClient was last updated four years ago, I went with ArduinoMqttClient by Arduino, currently v0.1.8 (beta). Along with WiFiClientSecure, I got mTLS MQTT with AWS IoT running successfully on ESP32.

What exactly isn't working? Just in case: the default encrypted port for MQTT is 8883, not 8884

I have to use w5500, ethernet module. cannot use WiFiClientSecure. try to use SSLClient lib and ESP_SSLClient lib.
both lib succeed on port 8883(encrypted). But when using client certificate nothing work so far

are there any methode to use WiFiClientSecure on w5500 ethernet lib?

One of the changes in the new v3 of arduino-esp32 is renaming WiFiClientSecure in v2 to NetworkClientSecure. Maybe the reason behind that is because it now works over Ethernet as well?

Version 3.0.1 was released ten days ago, and it's available as the esp32 Board by Espressif Systems. Haven't tried it myself.

will try...and inform you

I recommend you to use my new EthernetESP32 library which integrates with esp32 platform's version 3 networking, so you will use their NetworkClientSecure library with PubSubClient

will try too, thx

https://github.com/Networking-for-Arduino/EthernetESP32/blob/master/examples/WebClientSecure/WebClientSecure.ino

thx, plan to try it today

try below, but no succeed...

#include <EthernetESP32.h>
#include <NetworkClientSecure.h>
#include <PubSubClient.h>

unsigned long t,u;
#define DEV_ADDRESS 2172

W5500Driver driver;
//ENC28J60Driver driver;
//EMACDriver driver(ETH_PHY_LAN8720);

const char* mqttServer = "test.mosquitto.org"; // Broker address
const int mqttPort = 8884;// Broker port

// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(192, 168, 0, 177);
byte mac[] = { 0xDE, 0xED, 0xBA, 0xFE, 0xFE, 0xED };

const char *root_ca =
"-----BEGIN CERTIFICATE-----\n"
"MIIEAzCCAuugAwIBAgIUBY1hlCGvdj4NhBXkZ/uLUZNILAwwDQYJKoZIhvcNAQEL\n"
"BQAwgZAxCzAJBgNVBAYTAkdCMRcwFQYDVQQIDA5Vbml0ZWQgS2luZ2RvbTEOMAwG\n"
"A1UEBwwFRGVyYnkxEjAQBgNVBAoMCU1vc3F1aXR0bzELMAkGA1UECwwCQ0ExFjAU\n"
"BgNVBAMMDW1vc3F1aXR0by5vcmcxHzAdBgkqhkiG9w0BCQEWEHJvZ2VyQGF0Y2hv\n"
"by5vcmcwHhcNMjAwNjA5MTEwNjM5WhcNMzAwNjA3MTEwNjM5WjCBkDELMAkGA1UE\n"
"BhMCR0IxFzAVBgNVBAgMDlVuaXRlZCBLaW5nZG9tMQ4wDAYDVQQHDAVEZXJieTES\n"
"MBAGA1UECgwJTW9zcXVpdHRvMQswCQYDVQQLDAJDQTEWMBQGA1UEAwwNbW9zcXVp\n"
"dHRvLm9yZzEfMB0GCSqGSIb3DQEJARYQcm9nZXJAYXRjaG9vLm9yZzCCASIwDQYJ\n"
"KoZIhvcNAQEBBQADggEPADCCAQoCggEBAME0HKmIzfTOwkKLT3THHe+ObdizamPg\n"
"UZmD64Tf3zJdNeYGYn4CEXbyP6fy3tWc8S2boW6dzrH8SdFf9uo320GJA9B7U1FW\n"
"Te3xda/Lm3JFfaHjkWw7jBwcauQZjpGINHapHRlpiCZsquAthOgxW9SgDgYlGzEA\n"
"s06pkEFiMw+qDfLo/sxFKB6vQlFekMeCymjLCbNwPJyqyhFmPWwio/PDMruBTzPH\n"
"3cioBnrJWKXc3OjXdLGFJOfj7pP0j/dr2LH72eSvv3PQQFl90CZPFhrCUcRHSSxo\n"
"E6yjGOdnz7f6PveLIB574kQORwt8ePn0yidrTC1ictikED3nHYhMUOUCAwEAAaNT\n"
"MFEwHQYDVR0OBBYEFPVV6xBUFPiGKDyo5V3+Hbh4N9YSMB8GA1UdIwQYMBaAFPVV\n"
"6xBUFPiGKDyo5V3+Hbh4N9YSMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEL\n"
"BQADggEBAGa9kS21N70ThM6/Hj9D7mbVxKLBjVWe2TPsGfbl3rEDfZ+OKRZ2j6AC\n"
"6r7jb4TZO3dzF2p6dgbrlU71Y/4K0TdzIjRj3cQ3KSm41JvUQ0hZ/c04iGDg/xWf\n"
"+pp58nfPAYwuerruPNWmlStWAXf0UTqRtg4hQDWBuUFDJTuWuuBvEXudz74eh/wK\n"
"sMwfu1HFvjy5Z0iMDU8PUDepjVolOCue9ashlS4EB5IECdSR2TItnAIiIwimx839\n"
"LdUdRudafMu5T5Xma182OC0/u/xRlEm+tvKGGmfFcN0piqVl8OrSPBgIlb+1IKJE\n"
"m/XriWr/Cq4h/JfB7NTsezVslgkBaoU=\n"
"-----END CERTIFICATE-----\n";

const char *client_crt =
"-----BEGIN CERTIFICATE-----\n"
"MIIDqzCCApOgAwIBAgIBADANBgkqhkiG9w0BAQsFADCBkDELMAkGA1UEBhMCR0Ix\n"
"FzAVBgNVBAgMDlVuaXRlZCBLaW5nZG9tMQ4wDAYDVQQHDAVEZXJieTESMBAGA1UE\n"
"CgwJTW9zcXVpdHRvMQswCQYDVQQLDAJDQTEWMBQGA1UEAwwNbW9zcXVpdHRvLm9y\n"
"ZzEfMB0GCSqGSIb3DQEJARYQcm9nZXJAYXRjaG9vLm9yZzAeFw0yNDA2MTEwNTEw\n"
"MTlaFw0yNDA5MDkwNTEwMTlaMIGEMQswCQYDVQQGEwJJTjEOMAwGA1UECAwFamFi\n"
"YXIxDjAMBgNVBAcMBWJvZ29yMQ4wDAYDVQQKDAVyZW1vbjEQMA4GA1UECwwHcmVt\n"
"b2RldjEQMA4GA1UEAwwHYmludGFuZzEhMB8GCSqGSIb3DQEJARYSYmludGFuZ2hk\n"
"QHJlbW9uLmlkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwrmqg5Dj\n"
"7fjzpKvHxathwxI0pg3GOIjuL/PlMUhgc2Gw+wRNUo5a+bA1V9CaVX/UCmfxkM7e\n"
"u25Hr/7spKxSXiVakATBDZZ7rNtTVOgqjll6cre500oTav9/jNLMIBG4ajaFYIWq\n"
"F4GodMfHyBOPxoLEQFOjvyiFr9PnF3CEK0615F9Xl55b2ebULCO9EyCAthJkwoHk\n"
"HESETm2bngN1qEBkgUU2sAeLt4GdqbnRjFOZo5X7YCWtnbKhI/1NhnUXGRATw4Ml\n"
"TOLJh58ubrGT/kJfw/rzeIPuEA5dYDcwYiy2GMPwU6d6m+pWg+KyhQw7g8Kfodx2\n"
"YGfDCUarhltVPQIDAQABoxowGDAJBgNVHRMEAjAAMAsGA1UdDwQEAwIF4DANBgkq\n"
"hkiG9w0BAQsFAAOCAQEAAJVTcOgeeW5QdGeu1HNvRWBIl7CZdOWOAHcxsCuUM70l\n"
"ol+9EovFm9xJwicB/vKvAialyamxGzJaAHwYBgH/ATFiIWIb9Qo6on2pSGDCSOpA\n"
"OseQ2Y34qwfBFMphQSkpASbKwySVfsgQl82NcQKgVEod/SyWYQ7u5jsUd8nNd+vP\n"
"aN+CsyXKKXYCOUqMcBnFA40Bw5l8381VwvNcXe3RskbpW0fzsHh7GsC3WLUeLfsY\n"
"aJJWADOeogaaZiFG3pAizBZn74auTS0qHAf39z6Sfb5oFInb5OA7UcXJr5xMeMiW\n"
"dgYKSinUu78DWjaio5Yv+gjIhr6GfGT2+Z/b1jxo0A==\n"
"-----END CERTIFICATE-----\n";

const char *client_key =
"----BEGIN RSA PRIVATE KEY-----\n"
"MIIEowIBAAKCAQEAwrmqg5Dj7fjzpKvHxathwxI0pg3GOIjuL/PlMUhgc2Gw+wRN\n"
"Uo5a+bA1V9CaVX/UCmfxkM7eu25Hr/7spKxSXiVakATBDZZ7rNtTVOgqjll6cre5\n"
"00oTav9/jNLMIBG4ajaFYIWqF4GodMfHyBOPxoLEQFOjvyiFr9PnF3CEK0615F9X\n"
"l55b2ebULCO9EyCAthJkwoHkHESETm2bngN1qEBkgUU2sAeLt4GdqbnRjFOZo5X7\n"
"YCWtnbKhI/1NhnUXGRATw4MlTOLJh58ubrGT/kJfw/rzeIPuEA5dYDcwYiy2GMPw\n"
"U6d6m+pWg+KyhQw7g8Kfodx2YGfDCUarhltVPQIDAQABAoIBAF9eET3IW2k0URdo\n"
"LjoIWqPwKard+iO8JHOFzs0ZNmur/8Kgc8IoCDRbcXSlrMCND2tdRvj16C4Br8hp\n"
"YLqxztFfGACK44XlGoMpE6H05sq61faTNsZddC8mHPD6WJCoPRp/w0jnjctfbAJt\n"
"ydGaiwLdDoK2s+6uSPM5I+fd6k1Or5yKFEbCu2wDSfgy3Zs9JfcEmw/Ps0B3a29w\n"
"pJolVRZ8RPSq1KwTppk5chl19wFMAvKiOhQFjym/0QyTopfO7WC7qMkXqguudcGy\n"
"anvg22sOCyBNTK/Xe8mLRt9b9rAk46/CgxMXi+Yl0NO/9oBm892YQ2BX5j084K/8\n"
"KV7HFL0CgYEA4mVlYRwt5EEEN/KKad47l3fWqj1aoub9rsmH0jbyrRltZz+OU4iN\n"
"m1DU9t8WH7V70V6MGBKSVdF2/kwZxGQhkPWJsuvl8kth0Hio7wXKgdUCJm9wb+Gu\n"
"ZGMQbauFWBA9xdIAzrwTUfnLMp2fmJoMk5YXY24I3uADQkB4HBjZKrMCgYEA3DAX\n"
"C0zkcTHU9rkHrA5z4lOk00w1Nj99BcguV6tGe5e467fm4AlRMXPwF66erqGAHjpb\n"
"i/MoExuHXDTPRpaX+p7Sp6lb0gBLoIiVd7Vxoe8chdDNDnBt3voaCCeWdq/oamvE\n"
"ukratoJ50UwddA7MasoSXuqXH1g7v0i3Yz4AOE8CgYB9NuVuY16Cau9BXlTtnutI\n"
"qBBDUfHKPYW7YV1Mh/WKhZC71SwLyW/AomFXNSWQXhjgQLGtMsBa1KJCnaZ2t4bV\n"
"HOiO2+fvLYApBEMzB/nr+vrYdTQzvRduEFhqv73dagYwXijyLpVk9opLMWCuzyz7\n"
"h7zh0l73neBPiUalC504/wKBgDHplM+y9nXSXfwvpVWTUSGU5GAhfk51xdlxr0+g\n"
"O7H4iqqkXWmbsxQKu/oiU0YyxeAUSnxG+maabuyv6+yMZpchOY9YDMBl4L7bN0aT\n"
"ktaqZUTQOCt1wPIyr4P5tVniNkUpeMbiAQIuQX74ShTs5dmvRMrPUsKANcr+qNOO\n"
"Um5xAoGBAOJYrZ1o/bCon40t4asKqK7SQykLFXJdhc+WY6ENJN984Hl2oPnqC4d6\n"
"olVwxsJcitIFQ35nEx9/FZbUe95sQR1IHjNowe5w78Wy3O4THiFpNXbg2399MJg6\n"
"tLiV7QcuSaaRXhXm3mZrM7nv5+Q2qjtmHZY4vP+f+FRiZ8Mkd5LE\n"
"-----END RSA PRIVATE KEY-----\n";

void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");
for (int i=0;i<length;i++) {
Serial.print((char)payload[i]);
}
Serial.println();
}

NetworkClientSecure ethClientSSL;
PubSubClient client(mqttServer, mqttPort, callback, ethClientSSL); //pake certificate

void reconnect() {
// Loop until we're reconnected
while (!client.connected()) {
Serial.print("Attempting MQTT connection...");
// Attempt to connect
if (client.connect(String(DEV_ADDRESS).c_str(), "", "")) {
Serial.println("connected");
delay(2000);
// Once connected, publish an announcement...
client.publish("s2/1557/2172","hello world 0");
// ... and resubscribe
client.subscribe("s2/1557");
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 15 seconds");
// Wait 15 seconds before retrying
delay(15000);
}
}
}

void setup() {

Serial.begin(115200);
delay(500);
while (!Serial);

Ethernet.init(driver);

// start the Ethernet connection:
Serial.println("Initialize Ethernet with DHCP:");
if (Ethernet.begin() == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
// Check for Ethernet hardware present
if (Ethernet.hardwareStatus() == EthernetNoHardware) {
Serial.println("Ethernet shield was not found. Sorry, can't run without hardware. :(");
while (true) {
delay(1); // do nothing, no point running without Ethernet hardware
}
}
if (Ethernet.linkStatus() == LinkOFF) {
Serial.println("Ethernet cable is not connected.");
}
// try to configure using IP address instead of DHCP:
Ethernet.begin(ip);
} else {
Ethernet.begin(mac);
Serial.print(" DHCP assigned IP ");
Serial.println(Ethernet.localIP());
}

Serial.print("connecting to ");
Serial.print(mqttServer);
Serial.print(":");
Serial.print(mqttPort);
Serial.println("...");

//ethClientSSL.setInsecure(); // use only for a test!
ethClientSSL.setCACert(root_ca);
ethClientSSL.setCertificate(client_crt); // for client verification - certificate
ethClientSSL.setPrivateKey(client_key); // for client verification - private key

}

void loop() {
if (!client.connected()) {
reconnect();
}
client.loop();
if(millis()-t > 30000)
{
t=millis();
String message = "hello world "+String(u);
client.publish("s2/1557/2172",message.c_str());
Serial.println(message);
u++;
}
}

if change to port 8883, will work if disable setCertificate and setPrivateKey

NetworkClientSecure, it works for eth w5500

hi juraj, any suggestion? anyone?

setCertificate and setPrivateKey?
I don't know much about secure connections

ok thank you, anyone?

I change to ArduinoMqttClient and succeed with mTLS but on port 8883

than my question is test.mosquitto.org port 8884 really work?

  1. Use <CODE/> to post code. The syntax highlighting, monospaced fonts, and lack of word wrap can help people read your code and see problems.

    SPOILER: it is especially true in this case

  2. For big blocks of text, use C++ raw strings. For example, the root cert would start like

    const char *root_ca = R"(
    -----BEGIN CERTIFICATE-----
    MIIEAzCCAuugAwIBAgIUBY1hlCGvdj4NhBXkZ/uLUZNILAwwDQYJKoZIhvcNAQEL
    BQAwgZAxCzAJBgNVBAYTAkdCMRcwFQYDVQQIDA5Vbml0ZWQgS2luZ2RvbTEOMAwG
    

    The lack of lack of quotes and \n again makes the code more readable, and it's a snap to simply copy and paste the text as a file.

  3. When debugging on ESP32, compile with Debug output. In the bottom half of the Tools menu are the board-specific options. Core Level Debug defaults to None; change it to Debug, and re-upload.

  4. Now that you've broken the rule "Never Post Passwords and Private Keys", you'll need to do a new Certificate Signing Request to get a new cert and key.

OK, so with Debug enabled, with ArduinoMqttClient and NetworkClientSecure I see (wrapped here for readability)

[E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():282]:
(-15616) PK - Invalid key tag or value

That [E] indicates Error, which means with the least amount of debug output, Error, you would have seen this. More importantly, what does PK - Invalid key tag or value mean?

Well, let's take a step back and try some basic diagnostics: are the client cert and key valid? Do they match?

You probably have the original copies of the files downloaded, but you should take what is actually in your code. Maybe there was a copy&paste error. Here's where those raw strings would be handy, but you can still do it with what you have here, by stripping out the quotes and \n. With Linux (or WSL on Windows) you can use xsel, or pbpaste on a Mac; and openssl. First, for client_crt

xsel --clipboard --output \
| egrep -o '[A-Za-z0-9 /=+-]+' | egrep '..+' > client.crt
  1. Copy the two-line command, paste it into the shell, but don't press Enter.
  2. Next copy the big string into the clipboard.
  3. Then press Enter in the shell.

Repeat for client_key

xsel --clipboard --output \
| egrep -o '[A-Za-z0-9 /=+-]+' | egrep '..+' > client.key

Compare the modulus for both the public and private key. You will hash the values to make any differences obvious.

$ openssl x509 -modulus -noout -in client.crt | openssl md5
MD5(stdin)= e4fc41599618c82eb887048e8fdb261d

The private key uses rsa instead of x509

$ openssl rsa -modulus -noout -in client.key | openssl md5
Could not read private key from client.key
MD5(stdin)= d41d8cd98f00b204e9800998ecf8427e

Wait, there's an error. (That MD5 is not relevant; it's the hash of nothing; same as echo -n | openssl md5). Let's do an even more basic check.

$ file client.crt
client.crt: PEM certificate
$ file client.key
client.key: ASCII text

That shouldn't be just ASCII text. Take a closer look at the file. Big hint: the problem is on the first line

----BEGIN RSA PRIVATE KEY-----

There are only four hyphens on the left. There should be five. Fix the file and try again.

$ file client.key
client.key: PEM RSA private key
$ openssl rsa -modulus -noout -in client.key | openssl md5
MD5(stdin)= e4fc41599618c82eb887048e8fdb261d

OK, they match. Go back in the code

Fix it

const char *client_key =
"-----BEGIN RSA PRIVATE KEY-----\n"

Run it

[D][ssl_client.cpp:322] ssl_starttls_handshake():
Protocol is TLSv1.2 Ciphersuite is TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384
[D][ssl_client.cpp:324] ssl_starttls_handshake():
Record expansion is 29
You're connected to the MQTT broker!

Debug messages for a successful TLS connection -- this one with mTLS on port 8884, but it doesn't say that -- and the output from the MQTT example I used (File > Examples > ArduinoMqttClient > WiFiSimpleSender)

1 Like