MQTT and TLS

Projects Networking, Protocols, and Devices
21

feerdableusmoek:
Thanks. I've had a chance to look at that thread now and it seems focused on the client.setMutualAuthParams(mTLS); part which I think I have set ok. I think that, in part, because I am getting a very different error. The error I have is about the trust auhtority.

I am using a self-signed certificate, not a commercial certificate. So I am probably not converting the CA.crt (the certificate authority cert) and server.crt (the MQTT server's own certificate) files properly. I could certainly use more ideas about how to get both of those into the right format so they can be used by the SSLClient library. The library Example sketch shows three places marked // FIX ME but I am puzzled as to precisely what to put there in the three variables.

Post #8 did not help?

22

Idahowalker:
Post #8 did not help?

I think it did help because the error points to problems with the server's certificate.

23

That error probably means you are generating your Trust Anchors incorrectly (the CERTIFICATES_H_ file), as you suspected. These values can be generated by feeding the root certificate used by the server into the python tool included in SSLClient (SSLClient/pycert_bearssl.py at master · OPEnSLab-OSU/SSLClient · GitHub), which will extract the needed values and spit out the header file. More details on the problem and solution can be found in this issue comment (MQTT SSL possible? · Issue #7 · OPEnSLab-OSU/SSLClient · GitHub) and the SSLClient Trust Anchors documentation (SSLClient/TrustAnchors.md at master · OPEnSLab-OSU/SSLClient · GitHub). Does that help?

24

Load up Node-RED and use to see if one can create a MQTT connection to the server using the TLS cert files. Node-RED has been, for me, quite helpful in figuring out MQTT connection thingies. Node-RED can provide feedback of the connection process. Once a Node-RED connection is established coding for the connection become easier.

25

Thanks pileofwires and Idahowalker, I've pored over the Trust Anchor documents there yet again and while I do not have nodered around I do have Mosquitto working with the certificates. See one of the earlier posts above for the details on Mosquitto and the certificates. I've tried the brssl utility above in post #16 but seem to have missed a piece or mixed up something.

Breaking the question down into even smaller parts. The Trust Anchor documentation shows three variables:

  • static const unsigned char TA_DN0[]
  • static const unsigned char TA_RSA_N0[]
  • static const unsigned char TA_RSA_E0[]

For data,

  • certificate authority (named certificate-authority.crt)
  • server certificate (named server.crt)
  • server key (named server.key)

but which goes where in regards to the trust authority variables? And that third variable, TA_RSA_E0, seems undocumented. At least the guides do not make it clear. Again, these are all for a self-signed certificate

26

As mentioned I have some certificates which work over on a Raspberry Pi under GNU/Linux running Mosquitto as a client. When I try to import these using pycert_bearssl.py to create a certificates.h file, I have to modify the Python script a little:

$ diff pycert_bearssl.py modified_pycert_bearssl.py
140,148c140,141
<     if no_search:
<       root_certs = cert_objs
<     else:
<       for i, c in enumerate(cert_objs):
<         cn_hash = c.get_issuer().hash()
<         if cn_hash not in cert_dict:
<           click.echo('Could not find a root certificate for {0}'.format(cert[i].name))
<         else:
<           root_certs.append(cert_dict[cn_hash])
---
>     root_certs = cert_objs
>

Then it seems to produce a C data structure from the certificates (certificate-authority.crt and server.crt) or certificate (just server.crt or certificate-authority.crt):

./modified_pycert_bearssl.py server.crt

However, when I use that certificates.h file in my sketch, I get it complaining about hostnames. Here is output from the serial line:

Attempting MQTT connection...(SSLClient)(SSL_WARN)(connect): Arduino client is already connected? Continuing anyway...
(SSLClient)(SSL_WARN)(m_run_until): Terminating because the ssl engine closed
(SSLClient)(SSL_ERROR)(m_start_ssl): Failed to initlalize the SSL layer
(SSLClient)(SSL_ERROR)(m_print_br_error): Expected server name was not found in the chain.
failed, rc=-2 try again in 5 seconds

I can see in the header file that it appears to have found the certificate and the host name. At least the header contains a comment that it has found and processed the certificate:

...

/* This file is auto-generated by the pycert_bearssl tool.  Do not change it manually.
 * Certificates are BearSSL br_x509_trust_anchor format.  Included certs:
 *
 * Index:    0
 * Label:    192.168.1.57
 * Subject:  CN=192.168.1.57,L=Place,C=Country
 */
...

But maybe the problem is that I am using an IP number alone and not a fully qualified domain name?

I'm trying a different sketch modified from the Ethernet MQTT example:

#include <SPI.h>                  // built-in WiFi shield uses SPI
#include <WiFi101.h>              // WiFi support       https://www.arduino.cc/en/Reference/WiFi101
#include <WiFiUdp.h>

#include <SSLClient.h>            // https://github.com/OPEnSLab-OSU/SSLClient 
#include <PubSubClient.h>
#include "certificates.h"         // 
#include "wifi-secrets.h"         // 

char ssid[] = SECRET_SSID;        // network SSID
char pass[] = SECRET_PASS;        // network password
int status = WL_IDLE_STATUS;      // the WiFi radio's status

SSLClientParameters mTLS = SSLClientParameters::fromPEM(my_cert, sizeof my_cert, my_key, sizeof my_key);

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
}

WiFiClient foo;
SSLClient bar(foo, TAs, 2, A5);         // last value is Analog pin to draw random input from
PubSubClient mqttclient(mqttServer, 8883, callback, bar);

void reconnect() {
  // Loop until we're reconnected
  while (!mqttclient.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (mqttclient.connect("arduinoClient")) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      mqttclient.publish("outTopic", "hello world");
      // This is a workaround to address https://github.com/OPEnSLab-OSU/SSLClient/issues/9
      bar.flush();
      // ... and resubscribe
      mqttclient.subscribe("inTopic");
      // This is a workaround to address https://github.com/OPEnSLab-OSU/SSLClient/issues/9
      bar.flush();
    } else {
      Serial.print("failed, rc=");
      Serial.print(mqttclient.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
  // Start Serial
  WiFi.hostname("mkr1000");
  Serial.begin(115200);
  while (!Serial);

  // Enable mutual TLS with SSLClient
  bar.setMutualAuthParams(mTLS);

  WiFi.hostname("mkr1000");

  // check for the presence of the shield:
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present");
    // don't continue:
    while (true);
  }

  // attempt to connect to WiFi network:
  while ( WiFi.status() != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network:
    status = WiFi.begin(ssid, pass);

    // wait 5 seconds for connection:
    delay(5000);
  }


}

void loop() {
  if (!mqttclient.connected()) {
    reconnect();
  }
  mqttclient.loop();
}
27

Could you post the public key as you are using it? If the key is not formatted correctly and I figure the conversion program converts all the keys in the same fashion. So you should not be put out by posting the public key code.

28

Here are the contents of certificates.h from that last sketch:

// from mkr1000-client.crt
const char my_cert[] = "-----BEGIN CERTIFICATE-----\n\
MIIC/TCCAeUCFDgSCNPj6HFzyoqxmzmLoL14ORNeMA0GCSqGSIb3DQEBCwUAMDsx\n\
CzAJBgNVBAYTAkZJMRQwEgYDVQQHDAtSYXV0aW9zYWFyaTEWMBQGA1UEAwwNMTky\n\
LjE2OC4xLjE0MDAeFw0yMDA5MDcxMjA4MjRaFw0yMjAxMzExMjA4MjRaMDsxCzAJ\n\
BgNVBAYTAkZJMRQwEgYDVQQHDAtSYXV0aW9zYWFyaTEWMBQGA1UEAwwNMTkyLjE2\n\
OC4xLjExNjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAM6aMh1I93+X\n\
iYODPTWMbQo/PR8m8JdvU5XN28xBA0FOZImjg5BbJuCrDt3a4Xxu42S2enoMQwNv\n\
X8PoaMXeAexBuLlgTwLebU8Qr4rQ83z23xO5yT1n8HnJH/GxS2335TmAcdUBixL2\n\
QLvVaZEzWBxW4ZaVFrGjCS4BqsaHwFaP4qyV03pyUvWf5391MzIcPbpNDhHuP5q2\n\
HwME8ghReLq8u6FH9cHBSAHVcGUWGBMdtZGmY0ge7Onayfuwj3ecNP2SIoTSKcUk\n\
vrWkTsg1YzfQ4xH5e3lbjF7GGMsqJBA3ciWG8A8uCVTbEFhRqJI/pCGMaujskqOx\n\
MdH89+6cTasCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAPsrRGU4/cG+hLleqk55V\n\
eV3Y1UUsdEclqbKn84FIsv8i96NOJEzi1IXxzl/+oxpbYg2UxTeeNaDQ7ffyx8BG\n\
ehO+ymLjQ8OAz+VS41WXs6QQx2HLoeNCXbflo6/BaVhe3q6xqMLVm3Gm6rPUGs27\n\
bH8AJc+B444gUDo68sJEvm0M8w+teq7PhnRgqiIEpseE47IXu2OiVCU/dl4I8cEj\n\
6D/8QtDJ3F6sVhAkDEx908XMeIqU5eXQ0ZR3TfrWlP626Jq/4X99J6ZqyAFUgw4i\n\
miWmnGddy/2VSxpRlVUOPV1njUC6+nB3fFnDCGcSWFosELoudFGOYHbESNo4S1q7\n\
gw==\n\
-----END CERTIFICATE-----";

const char my_key[]  = "-----BEGIN RSA PRIVATE KEY-----\n\
MIIEpAIBAAKCAQEAzpoyHUj3f5eJg4M9NYxtCj89Hybwl29Tlc3bzEEDQU5kiaOD\n\
kFsm4KsO3drhfG7jZLZ6egxDA29fw+hoxd4B7EG4uWBPAt5tTxCvitDzfPbfE7nJ\n\
PWfweckf8bFLbfflOYBx1QGLEvZAu9VpkTNYHFbhlpUWsaMJLgGqxofAVo/irJXT\n\
enJS9Z/nf3UzMhw9uk0OEe4/mrYfAwTyCFF4ury7oUf1wcFIAdVwZRYYEx21kaZj\n\
SB7s6drJ+7CPd5w0/ZIihNIpxSS+taROyDVjN9DjEfl7eVuMXsYYyyokEDdyJYbw\n\
Dy4JVNsQWFGokj+kIYxq6OySo7Ex0fz37pxNqwIDAQABAoIBAGno6d/RU+74g+oD\n\
ERGuVwd1u6OIlrVKwJJ+4Ton6b9ErI1SYVNTJu7tQcXBOu5K+g7a7JY8Su4r3e/T\n\
6EHmuR2f405EbLyPwYFSkGqdCvT4qQq3ps3vxnrn+5QuyvgoqD199ty22dBypGSl\n\
1mQ06eo9L4++BdAyP7Zx7FWxqYu1cvs/0nCNJo0UFf00cTRyb4KVCsp0lCeq2vhj\n\
hC3XBQTLstglPHtesPKiPEXSWYiCy90nAlDgqVoDQA/DFFvMp8r3haV01aIQbtzn\n\
y1Ltog6AeAAR9rvDBbS+dAoUBnu5qHpZCqt7qzIn1J557/0oGNYovWrnVnXJJ9fx\n\
J8TCUoECgYEA8IK+DV4hHHZh85e+UrvSvzZsiRdWQK53S00TrME1yaSUHWdCWCJW\n\
geSEaYQ5OEC/4tKKp3zHFM1/W5AGNzEHgRDv18Lg0lIsmVSnPrUAgOVrah2OJqBJ\n\
Nt41/JT1RMxpY/jGGaGhgFw93zs1/B885acog4yjRw5sH+Xy/fAC+XECgYEA2+hp\n\
bI7dD6YVa/SO7gydxW8V6N11t3IezZcMJkT7DHpmvRqYvK1SeqLqcKxjFPTtl/tY\n\
VN1lHcw4m4nJOE+YXQ9Lc0nNiOVQWPiTLJVZHfhHXq1TSr73rvZV8uCG+wDewbs8\n\
Huep7XrM/ljj2Kbi/9cRPua4StfPZ85Idl46itsCgYEAwD3A665xiO6XHL4NwtOe\n\
gHhgCLQ+nTTW5bmWIF/PPn3uZhJDVW8c9nZR4IVvtycKOaNQcM+r0VTwl59nWHY4\n\
KVIAY9XBHlvQij6/7799x4mZchhtpzI3U0QKovntkjBRjTcsIRCwpokUa3pIjF3r\n\
ryUNnRis6f79LgH/WDqgJgECgYEAhaUwdOY6GOmRuUMuniOSustxIE1+3hb/+2gh\n\
3RwtCn9U8vY3/urC5n8eRY2uxay4vJXnL+RBMKPNf1F0tcFB0sSZ0R34SIs8kmW3\n\
Y+/78gfSsApcRV/bvTw+LdYgkmkL9+cneYzHK+lmrektx5xc9qko9PZY6DtR9D9c\n\
sHtBzuMCgYAAhdrekdzGcHNRxevaiJqmeY072TkWwQGvbuVoMd/PsVeYmVe/mbcS\n\
bEEghr9URmAMS4z6q50CyeuQvacxsw1iRdB5SEl2YLOnfhxj1Xx6+Yoz6sHCG2n0\n\
j0sGs6EtOkjSrqj3aEkfGhdvBC+GCTA6dXUGbbTgDMSxZ1+iYjqHZQ==\n\
-----END RSA PRIVATE KEY-----";

#ifndef _CERTIFICATES_H_
#define _CERTIFICATES_H_

#ifdef __cplusplus
extern "C"
{
#endif

/* This file is auto-generated by the pycert_bearssl tool.  Do not change it manually.
 * Certificates are BearSSL br_x509_trust_anchor format.  Included certs:
 *
 * Index:    0
 * Label:    192.168.1.140
 * Subject:  CN=192.168.1.140,L=Rautiosaari,C=FI
 */

#define TAs_NUM 1

static const unsigned char TA_DN0[] = {
    0x30, 0x3b, 0x31, 0x0b, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x06, 0x13,
    0x02, 0x46, 0x49, 0x31, 0x14, 0x30, 0x12, 0x06, 0x03, 0x55, 0x04, 0x07,
    0x0c, 0x0b, 0x52, 0x61, 0x75, 0x74, 0x69, 0x6f, 0x73, 0x61, 0x61, 0x72,
    0x69, 0x31, 0x16, 0x30, 0x14, 0x06, 0x03, 0x55, 0x04, 0x03, 0x0c, 0x0d,
    0x31, 0x39, 0x32, 0x2e, 0x31, 0x36, 0x38, 0x2e, 0x31, 0x2e, 0x31, 0x34,
    0x30,
};

static const unsigned char TA_RSA_N0[] = {
    0xd6, 0x34, 0x88, 0xf6, 0x8d, 0xcd, 0x5b, 0x21, 0xd7, 0x08, 0xd7, 0x78,
    0x6d, 0x93, 0x3c, 0x30, 0x06, 0x06, 0xc7, 0x7c, 0x9d, 0x63, 0xaf, 0x7d,
    0xd1, 0x5e, 0xc5, 0x38, 0xf2, 0xb8, 0x8c, 0xab, 0x70, 0x40, 0xd2, 0xab,
    0x98, 0x80, 0x1f, 0xcc, 0xa6, 0xd8, 0xba, 0xbf, 0x1a, 0x0d, 0xeb, 0x6e,
    0x57, 0x7e, 0xf9, 0x92, 0xf6, 0x92, 0xa1, 0xff, 0x25, 0xb3, 0x3b, 0x78,
    0xe5, 0x00, 0x58, 0x78, 0x2c, 0x41, 0xf2, 0xe1, 0xac, 0x36, 0xb3, 0x90,
    0x7e, 0x90, 0x92, 0x5c, 0x9e, 0x95, 0x77, 0xca, 0x23, 0x0c, 0xb8, 0xdd,
    0xc9, 0xe2, 0xff, 0xa2, 0x21, 0x10, 0xf5, 0x9a, 0xe5, 0xb8, 0x07, 0x8a,
    0xd8, 0xfa, 0xd5, 0xaa, 0x7c, 0x14, 0xf8, 0xb2, 0xb5, 0xb8, 0x6b, 0x43,
    0xb3, 0xff, 0x55, 0xa0, 0x58, 0x5b, 0x1d, 0xc1, 0x61, 0xe7, 0xdf, 0xac,
    0x17, 0xf9, 0x67, 0x8e, 0x06, 0xfa, 0x42, 0x11, 0x66, 0x72, 0x24, 0xdd,
    0x27, 0x92, 0x87, 0x87, 0x1a, 0x94, 0xfe, 0xde, 0x3f, 0xb6, 0x56, 0x68,
    0x11, 0xa2, 0xe7, 0xb1, 0x22, 0xa7, 0xea, 0xa1, 0xb1, 0x71, 0x6c, 0xc7,
    0x2a, 0xed, 0xd4, 0xef, 0xf9, 0xc0, 0xa4, 0x44, 0x66, 0x8b, 0x96, 0xae,
    0x22, 0x5c, 0xe9, 0x06, 0x39, 0x1e, 0x06, 0x55, 0x7e, 0x3a, 0x9b, 0xc2,
    0x3f, 0x65, 0x1c, 0xbb, 0x31, 0x73, 0x36, 0xa6, 0x27, 0x72, 0xe2, 0xa2,
    0xee, 0xe1, 0xd6, 0xfc, 0x55, 0xa3, 0xc6, 0xe8, 0x6c, 0x60, 0x2b, 0x8a,
    0xa2, 0xe5, 0x0f, 0x9b, 0x8d, 0xab, 0x6e, 0x22, 0xfe, 0x73, 0x30, 0x10,
    0xdc, 0x16, 0x2e, 0xdc, 0x85, 0xd9, 0xaf, 0xa7, 0xe0, 0x29, 0x4b, 0x30,
    0x14, 0xa4, 0x5a, 0x56, 0x18, 0xc8, 0xdb, 0x2b, 0xa8, 0x6a, 0x7c, 0x66,
    0xb2, 0xa7, 0xcc, 0xe7, 0xe1, 0x5e, 0x3b, 0xf4, 0x36, 0x50, 0xe7, 0x51,
    0x0a, 0x27, 0xd8, 0xd1,
};

static const unsigned char TA_RSA_E0[] = {
    0x01, 0x00, 0x01,
};

static const br_x509_trust_anchor TAs[] = {
    {
        { (unsigned char *)TA_DN0, sizeof TA_DN0 },
        BR_X509_TA_CA,
        {
            BR_KEYTYPE_RSA,
            { .rsa = {
                (unsigned char *)TA_RSA_N0, sizeof TA_RSA_N0,
                (unsigned char *)TA_RSA_E0, sizeof TA_RSA_E0,
            } }
        }
    },
};

#ifdef __cplusplus
} /* extern "C" */
#endif

#endif /* ifndef _CERTIFICATES_H_ */

Here are the same certificates as they are formatted for use by Mosquitto on the other system:

$ cat certificate-authority.crt 
-----BEGIN CERTIFICATE-----
MIIDVzCCAj+gAwIBAgIUKaLHUn/CYxO5sKL4NtKO8AzqBPcwDQYJKoZIhvcNAQEL
BQAwOzELMAkGA1UEBhMCRkkxFDASBgNVBAcMC1JhdXRpb3NhYXJpMRYwFAYDVQQD
DA0xOTIuMTY4LjEuMTQwMB4XDTIwMDgwNjEzMDM0M1oXDTIzMDUyNjEzMDM0M1ow
OzELMAkGA1UEBhMCRkkxFDASBgNVBAcMC1JhdXRpb3NhYXJpMRYwFAYDVQQDDA0x
OTIuMTY4LjEuMTQwMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1jSI
9o3NWyHXCNd4bZM8MAYGx3ydY6990V7FOPK4jKtwQNKrmIAfzKbYur8aDetuV375
kvaSof8lszt45QBYeCxB8uGsNrOQfpCSXJ6Vd8ojDLjdyeL/oiEQ9ZrluAeK2PrV
qnwU+LK1uGtDs/9VoFhbHcFh59+sF/lnjgb6QhFmciTdJ5KHhxqU/t4/tlZoEaLn
sSKn6qGxcWzHKu3U7/nApERmi5auIlzpBjkeBlV+OpvCP2UcuzFzNqYncuKi7uHW
/FWjxuhsYCuKouUPm42rbiL+czAQ3BYu3IXZr6fgKUswFKRaVhjI2yuoanxmsqfM
5+FeO/Q2UOdRCifY0QIDAQABo1MwUTAdBgNVHQ4EFgQUfe2qNAXmvkaODnNBV5eG
MT5DPwowHwYDVR0jBBgwFoAUfe2qNAXmvkaODnNBV5eGMT5DPwowDwYDVR0TAQH/
BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEASMRlsSFf93u55r46dxVV+QLwuzgv
xy+bZszUW9z+7QYlUtZ7DcmSDe743TuSQPFyArwq/JtNSd/2GtR9Eoim+bUXG+qi
ZObyUqDix+BP5gGiUwoNq5tlMGK7/+sZUe+e2T7SbCdUi770PNLZroOGzkDkqJbc
3dRG4uBUSBdzbt/aIY93cyaxhLVek6mHQshBQODyjIZklluqgaqxJzUlqVZbFwyf
V9HzFJuUA7goykB8eJfq0E6x9kPtfMeFmPnwpqt2WQye+7lLnSaLDU/KqukixrFk
WYSCIMWzXBZhspWqPpXxYHHbGi7lobTeOFKpseN4uPpah2hw5x/4aQVMMw==
-----END CERTIFICATE-----

$ cat server.crt 
-----BEGIN CERTIFICATE-----
MIIDFzCCAf+gAwIBAgIUOBII0+PocXPKirGbOYugvXg5E1kwDQYJKoZIhvcNAQEL
BQAwOzELMAkGA1UEBhMCRkkxFDASBgNVBAcMC1JhdXRpb3NhYXJpMRYwFAYDVQQD
DA0xOTIuMTY4LjEuMTQwMB4XDTIwMDgwNjEzMDM1NloXDTIyMDkxMjEzMDM1Nlow
OzELMAkGA1UEBhMCRkkxFDASBgNVBAcMC1JhdXRpb3NhYXJpMRYwFAYDVQQDDA0x
OTIuMTY4LjEuMTQwMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3blX
tIRgLkfdMSXmFVALeTje/OJdZtKmaZsAD3/jN9DhJzuWjtPuex3XD+ifuTEtsiSg
6oj3JUTxXte6rXZTtOHoqociiGg94YB46HlWXCXqC+Df1ccCekKC8uhtcAM1FOL8
LDkRK6Ihw5vZa9DXxu65VATlgtOIP/wwcUaBKSw+EEiibxoW+kcC6+Z85y4wOuBt
UIjYNmjSWpzLMLoS4aqO1SWEGGQchaCQmuE2+V1hc3tjH2zAWcWYBil9ZVJVf/yf
G3/0eiIf5ruQgu03t8f14pMnD8DwZv18eekfyPHH5y0gPQlgGtDe+pADqmcdbIBb
klGfxg6Mby/GntCWBwIDAQABoxMwETAPBgNVHREECDAGhwTAqAGMMA0GCSqGSIb3
DQEBCwUAA4IBAQBfD929QiZ9yMSlzFymC4bvllUWBBqMyuQdPMOwaJc9wlogKPnw
hEAj7QDo3iJUIFhs9dfjWTvC6Ja3s18Gv65Zt2vajk/yBYWKz8rCFPsmYTcZdC6p
KznBo3JFRtmVIBAldl9ysztOpd6by8zkAmKH0LCPN5UdWEC0cyXXq88xYt/H24M4
7YTlLD9kG1yIxGQ6c0rC3Qtq/TbaRtBz8C1g3OgSeekOrTd6nA7tIX61Je8EZwW3
ypJwAbXUJYjQwzgkGpahu9F6WOXNt96zXj5F3VIkOn41cCbYKSCsUurOIKFL9mV3
xGEd9B1Y+C7XeaNLhDwOU9TfuhD/pv8Ph217
-----END CERTIFICATE-----
29

Looks like

Perhaps my_cert = RSA_NO and my_key = one of the other TA_DNXXX thingies. Perhaps, because you did the conversions, you'll know which is which. almost finished documenting, moved SSLSession implementation into it'… · OPEnSLab-OSU/SSLClient@648104c · GitHub

30

Sorry I missed the part about the public keys:

$ cat certificate-authority.crt 
-----BEGIN CERTIFICATE-----
MIIDVzCCAj+gAwIBAgIUKaLHUn/CYxO5sKL4NtKO8AzqBPcwDQYJKoZIhvcNAQEL
BQAwOzELMAkGA1UEBhMCRkkxFDASBgNVBAcMC1JhdXRpb3NhYXJpMRYwFAYDVQQD
DA0xOTIuMTY4LjEuMTQwMB4XDTIwMDgwNjEzMDM0M1oXDTIzMDUyNjEzMDM0M1ow
OzELMAkGA1UEBhMCRkkxFDASBgNVBAcMC1JhdXRpb3NhYXJpMRYwFAYDVQQDDA0x
OTIuMTY4LjEuMTQwMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1jSI
9o3NWyHXCNd4bZM8MAYGx3ydY6990V7FOPK4jKtwQNKrmIAfzKbYur8aDetuV375
kvaSof8lszt45QBYeCxB8uGsNrOQfpCSXJ6Vd8ojDLjdyeL/oiEQ9ZrluAeK2PrV
qnwU+LK1uGtDs/9VoFhbHcFh59+sF/lnjgb6QhFmciTdJ5KHhxqU/t4/tlZoEaLn
sSKn6qGxcWzHKu3U7/nApERmi5auIlzpBjkeBlV+OpvCP2UcuzFzNqYncuKi7uHW
/FWjxuhsYCuKouUPm42rbiL+czAQ3BYu3IXZr6fgKUswFKRaVhjI2yuoanxmsqfM
5+FeO/Q2UOdRCifY0QIDAQABo1MwUTAdBgNVHQ4EFgQUfe2qNAXmvkaODnNBV5eG
MT5DPwowHwYDVR0jBBgwFoAUfe2qNAXmvkaODnNBV5eGMT5DPwowDwYDVR0TAQH/
BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEASMRlsSFf93u55r46dxVV+QLwuzgv
xy+bZszUW9z+7QYlUtZ7DcmSDe743TuSQPFyArwq/JtNSd/2GtR9Eoim+bUXG+qi
ZObyUqDix+BP5gGiUwoNq5tlMGK7/+sZUe+e2T7SbCdUi770PNLZroOGzkDkqJbc
3dRG4uBUSBdzbt/aIY93cyaxhLVek6mHQshBQODyjIZklluqgaqxJzUlqVZbFwyf
V9HzFJuUA7goykB8eJfq0E6x9kPtfMeFmPnwpqt2WQye+7lLnSaLDU/KqukixrFk
WYSCIMWzXBZhspWqPpXxYHHbGi7lobTeOFKpseN4uPpah2hw5x/4aQVMMw==
-----END CERTIFICATE-----

$ cat server.crt 
-----BEGIN CERTIFICATE-----
MIIDFzCCAf+gAwIBAgIUOBII0+PocXPKirGbOYugvXg5E1kwDQYJKoZIhvcNAQEL
BQAwOzELMAkGA1UEBhMCRkkxFDASBgNVBAcMC1JhdXRpb3NhYXJpMRYwFAYDVQQD
DA0xOTIuMTY4LjEuMTQwMB4XDTIwMDgwNjEzMDM1NloXDTIyMDkxMjEzMDM1Nlow
OzELMAkGA1UEBhMCRkkxFDASBgNVBAcMC1JhdXRpb3NhYXJpMRYwFAYDVQQDDA0x
OTIuMTY4LjEuMTQwMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3blX
tIRgLkfdMSXmFVALeTje/OJdZtKmaZsAD3/jN9DhJzuWjtPuex3XD+ifuTEtsiSg
6oj3JUTxXte6rXZTtOHoqociiGg94YB46HlWXCXqC+Df1ccCekKC8uhtcAM1FOL8
LDkRK6Ihw5vZa9DXxu65VATlgtOIP/wwcUaBKSw+EEiibxoW+kcC6+Z85y4wOuBt
UIjYNmjSWpzLMLoS4aqO1SWEGGQchaCQmuE2+V1hc3tjH2zAWcWYBil9ZVJVf/yf
G3/0eiIf5ruQgu03t8f14pMnD8DwZv18eekfyPHH5y0gPQlgGtDe+pADqmcdbIBb
klGfxg6Mby/GntCWBwIDAQABoxMwETAPBgNVHREECDAGhwTAqAGMMA0GCSqGSIb3
DQEBCwUAA4IBAQBfD929QiZ9yMSlzFymC4bvllUWBBqMyuQdPMOwaJc9wlogKPnw
hEAj7QDo3iJUIFhs9dfjWTvC6Ja3s18Gv65Zt2vajk/yBYWKz8rCFPsmYTcZdC6p
KznBo3JFRtmVIBAldl9ysztOpd6by8zkAmKH0LCPN5UdWEC0cyXXq88xYt/H24M4
7YTlLD9kG1yIxGQ6c0rC3Qtq/TbaRtBz8C1g3OgSeekOrTd6nA7tIX61Je8EZwW3
ypJwAbXUJYjQwzgkGpahu9F6WOXNt96zXj5F3VIkOn41cCbYKSCsUurOIKFL9mV3
xGEd9B1Y+C7XeaNLhDwOU9TfuhD/pv8Ph217
-----END CERTIFICATE-----

Idahowalker:
Perhaps, because you did the conversions, you'll know which is which

The my_cert and my_key are for the client for mTLS, up near the start of the sketch.

The server certificate and the CA seem OK:

openssl x509 -in certificate-authority.crt -text
openssl x509 -in server.crt -text

As for the Trust Authority array, I've tried all four combinations to generate certificates.h

./modified_pycert_bearssl.py certificate-authority.crt server.crt
./modified_pycert_bearssl.py server.crt certificate-authority.crt
./modified_pycert_bearssl.py certificate-authority.crt
./modified_pycert_bearssl.py server.crt

and still get the same error

Attempting MQTT connection...(SSLClient)(SSL_WARN)(connect): Arduino client is already connected? Continuing anyway...
(SSLClient)(SSL_WARN)(m_run_until): Terminating because the ssl engine closed
(SSLClient)(SSL_ERROR)(m_start_ssl): Failed to initlalize the SSL layer
(SSLClient)(SSL_ERROR)(m_print_br_error): Expected server name was not found in the chain.

I'm suspecting that one of the main problems is the use of an IP address and not a FQDN. However, not sure how to get around that at the moment. Like mentioned, I have Mosquitto working with this server certificate and certificate authority over on a Raspberry Pi.

31

Could you try regenerating your certificates with a proper FQDN (like in this tutorial: Creating and Using Client Certificates with MQTT and Mosquitto)? It looks like you're generating the trust anchors correctly, so I suspect you are correct that an improper common name for your certificates is the issue. You may also try giving the CA a different common name than the server, since having them both be the same might cause issues in BearSSL's validation engine. Finally, I would make sure that you are specifying the MQTT server IP address in your sketch as an IPAddress object (instead of a string) so SSLClient doesn't accidentally treat it as a domain and attempt to verify it (more details in this issue comment: ESP32 W5500 MQTT SSL · Issue #8 · OPEnSLab-OSU/SSLClient · GitHub).

32

Here were the steps I've taken to get the current set of keys and certificates:

read -p "Make certificate authority and corresponding key " a
 openssl req \
        -new \
        -x509 \
        -days 1023 \
        -extensions v3_ca \
        -subj '/C=FI/L=Here/CN=198.51.100.14' \
        -keyout certificate-authority.key \
        -out    certificate-authority.crt 

echo
read -p "Make server key" a
# Server Key (signed)

 openssl genrsa \
        -out server.key \
        2048

echo
read -p "Make server certificate signing request" a
 openssl req \
        -new \
        -out server.csr \
        -key server.key \
        -subj '/C=FI/L=Here/CN=198.51.100.14'

echo
read -p "Sign server key" a
 openssl x509 \
        -req \
        -days 767 \
        -CA    certificate-authority.crt \
        -CAkey certificate-authority.key \
        -CAcreateserial \
        -in  server.csr \
        -out server.crt \
        -extfile <(printf "subjectAltName=IP:198.51.100.14")

I'm not sure if I've done the subjectAltName correctly or if it is sufficient to work with SSLClient and self-signed certificates.

This network is appears to be a proprietary, stripped down version of OpenWRT and might not be flexible enough to assign multiple names or IP numbers to a single device. It is a constraining factor. However, I will look into setting a FQDN and trying a new batch of keys and certificates with differing names, if possible, and maybe different IP addresses for the CA and the server certificate.

Right now, the sketch produces the following output,

Setup: Attempting to connect to WPA SSID: xxxxxxxx
SSID: xxxxxxxx
BSSID: xx:xx:xx:xx:xx:xx
signal strength (RSSI): -65
Encryption Type: 2

Setup is done
Reconnect() attempting MQTT connection...(SSLClient)(SSL_WARN)(connect): Using a raw IP Address for an SSL connection bypasses some important verification steps. You should use a domain name (www.google.com) whenever possible.
failed, rc=5 try again in 5 seconds
Reconnect() attempting MQTT connection...(SSLClient)(SSL_WARN)(connect): Using a raw IP Address for an SSL connection bypasses some important verification steps. You should use a domain name (www.google.com) whenever possible.
failed, rc=5 try again in 5 seconds
. . .

and I see the following type if log messages in the Mosquitto broker's log,

. . .
1599994443: New connection from 198.51.100.16 on port 8883.
1599994443: Socket error on client <unknown>, disconnecting.
1599994448: New connection from 198.51.100.16 on port 8883.
1599994449: Socket error on client <unknown>, disconnecting.
. . .
33

Substantial progress. The client now connects:

Setup: Attempting to connect to WPA SSID: xxxxxxx
SSID: xxxxxxx
BSSID: xx:xx:xx:xx:xx:xx
signal strength (RSSI): -56
Encryption Type: 2

Setup is done
Reconnect() attempting MQTT connection...(SSLClient)(SSL_WARN)(connect): Using a raw IP Address for an SSL connection bypasses some important verification steps. You should use a domain name (www.google.com) whenever possible.
connected
(SSLClient)(SSL_ERROR)(m_update_engine): Error writing to m_client
(SSLClient)(SSL_ERROR)(m_update_engine): 1
(SSLClient)(SSL_ERROR)(connected): Socket was unexpectedly interrupted. m_client error: 
(SSLClient)(SSL_ERROR)(connected): 1
(SSLClient)(SSL_ERROR)(connected): Not connected because write error is set
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL
(SSLClient)(SSL_WARN)(m_run_until): Terminating with write error: 
(SSLClient)(SSL_WARN)(m_run_until): 4
(SSLClient)(SSL_ERROR)(flush): Could not flush write buffer!
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL
(SSLClient)(SSL_ERROR)(connected): Not connected because write error is set
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL
(SSLClient)(SSL_ERROR)(connected): Not connected because write error is set
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL
(SSLClient)(SSL_ERROR)(connected): Not connected because write error is set
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL
Reconnect() attempting MQTT connection...(SSLClient)(SSL_ERROR)(connected): Not connected because write error is set
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL
(SSLClient)(SSL_ERROR)(connected): Not connected because write error is set
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL
(SSLClient)(SSL_WARN)(connect): Using a raw IP Address for an SSL connection bypasses some important verification steps. You should use a domain name (www.google.com) whenever possible.
connected

with this sketch and new keys / certificates.

#include <SPI.h>                  // built-in WiFi shield uses SPI
#include <WiFi101.h>              // WiFi support       https://www.arduino.cc/en/Reference/WiFi101
#include <WiFiUdp.h>

#include <SSLClient.h>            // https://github.com/OPEnSLab-OSU/SSLClient 
#include <PubSubClient.h>         // https://pubsubclient.knolleary.net/ 

#include "certificates.h"         // 
#include "wifi-secrets.h"         // 

char ssid[] = SECRET_SSID;        // network SSID
char pass[] = SECRET_PASS;        // network password
int status  = WL_IDLE_STATUS;     // the WiFi radio's status

SSLClientParameters mTLS = SSLClientParameters::fromPEM(my_cert, sizeof my_cert, my_key, sizeof my_key);

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
}

void printMacAddress(byte mac[]) {
  for (int i = 5; i >= 0; i--) {
    if (mac[i] < 16) {
      Serial.print("0");
    }
    Serial.print(mac[i], HEX);
    if (i > 0) {
      Serial.print(":");
    }
  }
  Serial.println();
}

void printCurrentNet() {
  // print the SSID of the network you're attached to:
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // print the MAC address of the router you're attached to:
  byte bssid[6];
  WiFi.BSSID(bssid);
  Serial.print("BSSID: ");
  printMacAddress(bssid);

  // print the received signal strength:
  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI): ");
  Serial.println(rssi);

  // print the encryption type:
  byte encryption = WiFi.encryptionType();
  Serial.print("Encryption Type: ");
  Serial.println(encryption, HEX);
  Serial.println();
}

WiFiClient wifi;
SSLClient  tls(wifi, TAs, 2, A5);         // the last value is an Analog pin to draw random input from

PubSubClient mqttClient(mqttServer, mqttPort, callback, tls);
// mqttClient.setServer(mqttServer, mqttPort);

void reconnect() {
  // Loop until we're reconnected
  while (!mqttClient.connected()) {
    Serial.print("Reconnect() attempting MQTT connection...");
    // Attempt to connect
    if (mqttClient.connect("mkr1000", MQTT_ACCT, MQTT_PASS)) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      mqttClient.publish("outTopic", "hello world");
      // This is a workaround to address https://github.com/OPEnSLab-OSU/SSLClient/issues/9
      tls.flush();
      // ... and resubscribe
      mqttClient.subscribe("inTopic");
      // This is a workaround to address https://github.com/OPEnSLab-OSU/SSLClient/issues/9
      tls.flush();
    } else {
      Serial.print("failed, rc=");
      Serial.print(mqttClient.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
  // Start Serialw
  WiFi.hostname("mkr1000");
  Serial.begin(115200);
  while (!Serial);

  // Enable mutual TLS with SSLClient
  tls.setMutualAuthParams(mTLS);

  WiFi.hostname("mkr1000");

  // check for the presence of the shield:
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("Setup: WiFi shield not present");
    // don't continue:
    while (true);
  }

  // attempt to connect to WiFi network:
  while ( WiFi.status() != WL_CONNECTED) {
    Serial.print("Setup: Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network:
    status = WiFi.begin(ssid, pass);

    // wait 5 seconds for connection:
    delay(5000);
  }

  printCurrentNet();

  Serial.println("Setup is done");
}

void loop() {
  if (!mqttClient.connected()) {
    reconnect();
  }
  mqttClient.loop();
}

(Edit: here is a log excerpt from the MQTT broker, showing that it does connect:

1600146220: New connection from xx.xx.xx.xx on port 8883.
1600146220: New client connected from xx.xx.xx.xx as mkr1000 (c1, k15, u'foo').
1600146242: Client mkr1000 has exceeded timeout, disconnecting.
1600146242: Socket error on client mkr1000, disconnecting.

I'll look at PubSubClient.h more closely now.

34

Try.

void reconnect() {
   mqttClient.setKeepAlive( 90 );
  // Loop until we're reconnected
  while (!mqttClient.connected()) {

I did not see where a callback is defined?

35

Clues:

#include <WiFi.h>
#include <PubSubClient.h>
#include "certs.h"
#include "sdkconfig.h"
#include "esp_system.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/timers.h"
#include "freertos/event_groups.h"
#include <SPI.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BME280.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1351.h>
#include "esp32-hal-psram.h"
#include "esp_himem.h"
#include "time.h"
//
//
hw_timer_t * timer = NULL;
EventGroupHandle_t eg;
#define evtDoDisplay ( 1 << 1 )
#define evtCollectHistory ( 1 << 2 )
#define evtParseMQTT ( 1 << 5 )
#define evtDoTheBME280Thing ( 1 << 8 )
#define evtDoTheHumidityThing ( 1 << 9 )
#define evtDoTheSunLampThing ( 1 << 10 )
#define evtMQTT_Disconnect ( 1 << 11 )
// the tevents in this event groups tasks have been given a delay to avoid a possible race condition.
#define evtGroupBits ( evtCollectHistory | evtDoTheBME280Thing | evtDoTheHumidityThing | evtDoTheSunLampThing | evtDoDisplay )
//
Adafruit_SSD1351 tft = Adafruit_SSD1351( 128, 128, GPIO_NUM_27, GPIO_NUM_12, GPIO_NUM_13, GPIO_NUM_14, GPIO_NUM_26 );
////
Adafruit_BME280 bme( GPIO_NUM_5 ); // hardware SPI
////
WiFiClient wifiClient;
PubSubClient MQTTclient( mqtt_server, mqtt_port, wifiClient );
////
////
// memory pointers for variables stored in himem, WROVER PSRAM.
/*
   ePtr = envirtonemtal pointers to float values
   [0] = inside temperature
   [1] = inside humidity
   {2} = inside pressure
   [3] = outside temperature
   [4] = outside humidity
   [5] = outside pressure
   [6] = outside AQ index
*/
float *ePtr;
float *oPressureHistoryPtr;
float *oIAQ_HistoryPtr;
float *oHumidityHistoryPtr;
float *oTemperaturePtr;
////
char *strPayloadPtr;
char *str_eTopicPtr;
////
////
int *ColorPalettePtr;
/*
   int data pointer, used to store globals
   IntDataPtr[0] int DataCells = 50
   IntDataPtr[1] int arrayCellPtr = 0;
   IntDataPtr[2] int humidity set point
   IntDataPtr[3] humidity control enable
   IntDataPtr[4] = sunlamp on
   IntDataPtr[5] = sun lamp enable
*/
int *IntDataPtr;
////
byte mac[6];
////
// RTC_DATA_ATTR int bootCount = 0; // assign a memory location in RTC FAST RAM, an experiment remnent
////
////
SemaphoreHandle_t sema_HistoryCompleted;
SemaphoreHandle_t sema_MQTT_Parser;;
SemaphoreHandle_t sema_TheHumidityThing;
SemaphoreHandle_t sema_DoTheSunLampTHing;
SemaphoreHandle_t sema_MQTT_KeepAlive;
////
volatile int iDoTheBME280Thing = 0;
////
void IRAM_ATTR onTimer()
{
  BaseType_t xHigherPriorityTaskWoken;
  iDoTheBME280Thing++;
  if ( iDoTheBME280Thing == 60000 )
  {
    xEventGroupSetBitsFromISR( eg, evtGroupBits, &xHigherPriorityTaskWoken );
    iDoTheBME280Thing = 0;
  }
}
36 
////
void setup()
{
  //
  gpio_config_t io_cfg = {};
  io_cfg.mode = GPIO_MODE_OUTPUT;
  //bit mask of the pins to set
  io_cfg.pin_bit_mask = ( (1ULL << GPIO_NUM_0) | (1ULL << GPIO_NUM_15) | (1ULL << GPIO_NUM_2) );
  //configure GPIO with the given settings
  gpio_config(&io_cfg);
  REG_WRITE(GPIO_OUT_W1TS_REG, BIT0);
  REG_WRITE(GPIO_OUT_W1TC_REG, BIT12);
  REG_WRITE(GPIO_OUT_W1TC_REG, BIT15);
  //
  /* Use 4th timer of 4.
    1 tick 1/(80MHZ/80) = 1us set divider 80 and count up.
    Attach onTimer function to timer
    Set alarm to call timer ISR, every 1000uS and repeat / reset ISR (true) after each alarm
    Start an timer alarm
  */
  timer = timerBegin( 3, 80, true );
  timerAttachInterrupt( timer, &onTimer, true );
  timerAlarmWrite(timer, 1000, true);
  timerAlarmEnable(timer);
  //
  // memory allocations for items to be stored in PSRAM
  log_i("before Free PSRAM: %d", ESP.getFreePsram());
  IntDataPtr = (int*)ps_calloc( 15, sizeof(int) );
  IntDataPtr[0] = 60; //const int DataCells = 60;
  oPressureHistoryPtr = (float*)ps_calloc( IntDataPtr[0], sizeof(float) );// oPressureHistoryPtr = (float*)ps_calloc( DataCells, sizeof(float) );
  oIAQ_HistoryPtr = (float*)ps_calloc( IntDataPtr[0], sizeof(float) );//oIAQ_HistoryPtr = (float*)ps_calloc( DataCells, sizeof(float) );
  oHumidityHistoryPtr = (float*)ps_calloc( IntDataPtr[0], sizeof(float) );
  oTemperaturePtr = (float*)ps_calloc( IntDataPtr[0], sizeof(float) );
  ePtr =  (float*)ps_calloc( 15, sizeof(float) );
  strPayloadPtr = (char *)ps_calloc(300, sizeof(char) );
  for ( int i = 0; i < 299; i++ )
  {
    strPayloadPtr[i] = '\0';
    strPayloadPtr[i] = '\0';
  }
  str_eTopicPtr = (char *)ps_calloc(300, sizeof(char) );
  ColorPalettePtr  = (int*)ps_calloc( 10, sizeof(int) );
  log_i("Total heap: %d", ESP.getHeapSize());
  log_i("Free heap: %d", ESP.getFreeHeap());
  log_i("Total PSRAM: %d", ESP.getPsramSize());
  log_i("Free PSRAM: %d", ESP.getFreePsram());
  // popuate color palette for display use
  ColorPalettePtr[0] = 0x0000; //BLACK
  ColorPalettePtr[1] = 0x001F; //BLUE
  ColorPalettePtr[2] = 0xF800; //RED
  ColorPalettePtr[3] = 0x07E0; //GREEN
  ColorPalettePtr[4] = 0x07FF; //CYAN
  ColorPalettePtr[5] = 0xF81F; //MAGENTA
  ColorPalettePtr[6] = 0xFFE0; //YELLOW
  ColorPalettePtr[7] = 0xFFFF; //WHITE
  ColorPalettePtr[8] = 0xFFFFD8; //LIGHTYELLOW
  ColorPalettePtr[9] = 0xFF8040; //BROWN
  //
  ePtr[2] = 50; // set a default humidity level
  ePtr[3] = 0; // set humidity control to off
  //
  eg = xEventGroupCreate();
  SPI.begin();
  bme.begin();
  while ( !bme.begin() )
  {
    log_i( "Waiting on response from BME280");
    vTaskDelay( 1000 );
  }
  tft.begin();
  tft.fillScreen(0x0000);
  //
  sema_MQTT_Parser = xSemaphoreCreateBinary();
  sema_HistoryCompleted = xSemaphoreCreateBinary();
  sema_TheHumidityThing = xSemaphoreCreateBinary();
  sema_DoTheSunLampTHing = xSemaphoreCreateBinary();
  sema_MQTT_KeepAlive = xSemaphoreCreateBinary();
  xSemaphoreGive( sema_DoTheSunLampTHing );
  xSemaphoreGive( sema_HistoryCompleted );
  xSemaphoreGive( sema_TheHumidityThing );
  xSemaphoreGive( sema_MQTT_KeepAlive ); // found keep alive can mess with a publish, stop keep alive during publish
  ////
  xTaskCreatePinnedToCore( fparseMQTT, "fparseMQTT", 7000, NULL, 5, NULL, 1 ); // assign all to core 1, WiFi in use.
  xTaskCreatePinnedToCore( fCollectHistory, "fCollectHistory", 10000, NULL, 4, NULL, 1 );
  xTaskCreatePinnedToCore( MQTTkeepalive, "MQTTkeepalive", 7000, NULL, 2, NULL, 1 ); //this task makes a WiFi and MQTT connection.
  xTaskCreatePinnedToCore( fUpdateDisplay, "fUpdateDisplay", 50000, NULL, 5, NULL, 1 );
  xTaskCreatePinnedToCore( DoTheBME280Thing, "DoTheBME280Thing", 20000, NULL, 3, NULL, 1 );
  xTaskCreatePinnedToCore( fDoTheHumidityThing, "fDoTheHumidityThing", 2000, NULL, 3, NULL, 1 );
  xTaskCreatePinnedToCore( fDoTheSunLampThing, "fDoTheSunLampThing", 2000, NULL, 3, NULL, 1 );
  //xTaskCreatePinnedToCore( fMQTT_Disconnect, "fMQTT_Disconnect", 2000, NULL, 2, NULL, 1 );
  ////
} //setup()
////
////
// disconnect causes the MQTT data presistance to clear the presisted data. outside environmental data is updated once every 5 minutes.
void fMQTT_Disconnect( void * parameter )
{
  int i = 0;
  for (;;)
  {
    xEventGroupWaitBits (eg, evtMQTT_Disconnect, pdTRUE, pdTRUE, portMAX_DELAY );
    //    if ( i >= 60 )
    //    {
    if ( i >= 10 )
    {
      xSemaphoreTake( sema_MQTT_KeepAlive, portMAX_DELAY ); //
      MQTTclient.disconnect();
      xSemaphoreGive( sema_MQTT_KeepAlive );
      i = 0;
    } else {
      i++;
    }
    log_i( "discconect count %d", i );
  }
  vTaskDelete( NULL );
}
////
void GetTheTime()
{
  char* ntpServer = "2.us.pool.ntp.org";
  int gmtOffset_sec = -(3600 * 7 );
  int daylightOffset_sec = 3600;
  configTime(gmtOffset_sec, daylightOffset_sec, ntpServer);
}
////
// http://www.cplusplus.com/reference/ctime/strftime/
////
int getHour()
{
  struct tm timeinfo;
  getLocalTime(&timeinfo);
  char _time[ 5 ];
  strftime( _time, 80, "%T", &timeinfo );
  return String(_time).toInt();
}
////
void printLocalTime() {
  struct tm timeinfo;
  getLocalTime(&timeinfo);
  char _time[ 80 ];
  strftime( _time, 80, "%T", &timeinfo );
  log_i( "%s", _time);
  //  }
}
////
void fDoTheSunLampThing( void * parameter )
{
  //int x = gpio_get_level( GPIO_NUM_15 ); // reads gpio pin state returns an int
  // IntDataPtr[4] = sunlamp on manual mode, automatic mode off for manual mode to work
  // IntDataPtr[5] = sun lamp enable automatic mode
  bool AlreadyOn = false;
  for (;;)
  {
    xEventGroupWaitBits (eg, evtDoTheSunLampThing, pdTRUE, pdTRUE, portMAX_DELAY );
    vTaskDelay( 6 );
    xSemaphoreTake( sema_DoTheSunLampTHing, portMAX_DELAY );
    // sun lamp on/off auto disabled
    if ( (IntDataPtr[4] == 1) && (IntDataPtr[5] == 0)  )
    {
      if ( !AlreadyOn )
      {
        REG_WRITE(GPIO_OUT_W1TS_REG, BIT15);
        AlreadyOn = true;
      } else {
        REG_WRITE(GPIO_OUT_W1TC_REG, BIT15);
        AlreadyOn = false;
      }
    } else if ( (IntDataPtr[4] == 0) && (IntDataPtr[5] == 1) ) // light off auto enabled
    {
      int _hour = getHour();
      if ( (_hour >= 7) && (_hour <= 17) )
      {
        REG_WRITE(GPIO_OUT_W1TS_REG, BIT15);
        AlreadyOn = true;
      }
      if ( (_hour < 7) || (_hour > 16) )
      {
        REG_WRITE(GPIO_OUT_W1TC_REG, BIT15);
        AlreadyOn = false;
      }
    } else {
      REG_WRITE(GPIO_OUT_W1TC_REG, BIT15);
      AlreadyOn = false;
    }
    xSemaphoreGive( sema_DoTheSunLampTHing );
    //log_i( "fDoTheSunLampThing high watermark %d",  uxTaskGetStackHighWaterMark( NULL ) );
  }
  vTaskDelete( NULL );
} //void fCollectHistory( void * parameter )
////
// send a signal out to relay to energize or de-energize humidifier based on set point
void fDoTheHumidityThing( void * parameter )
{
  for (;;)
  {
    xEventGroupWaitBits (eg, evtDoTheHumidityThing, pdTRUE, pdTRUE, portMAX_DELAY );
    vTaskDelay( 4 );
    xSemaphoreTake( sema_TheHumidityThing, portMAX_DELAY );
    if ( IntDataPtr[3] == 1 )
    {
      if ( IntDataPtr[2] + 1 < (int)ePtr[2] )
      {
        REG_WRITE(GPIO_OUT_W1TC_REG, BIT2);
      }
      if ( (IntDataPtr[2] - 1) > (int)ePtr[2] )
      {
        // energize humidifier
        REG_WRITE(GPIO_OUT_W1TS_REG, BIT2);
      }
    } else {
      //de energize humidifier
      REG_WRITE(GPIO_OUT_W1TC_REG, BIT2);
    }
    xSemaphoreGive( sema_TheHumidityThing );
    //log_i( "fDoTheHumidityThing high watermark %d",  uxTaskGetStackHighWaterMark( NULL ) );
  }
  vTaskDelete( NULL );
} //void fCollectHistory( void * parameter )
////
37 
/*
   Collect history information
   task triggered by hardware timer once a minute
   stores history into PSRAM
   Using a semaphore to protect the PSRAM from multiple tasks access the same PSRM locations
*/
void fCollectHistory( void * parameter )
{
  int StorageTriggerCount = 119;
  bool Tick = true;
  int maxStorageTriggerCount = 120;
  for (;;)
  {
    xEventGroupWaitBits (eg, evtCollectHistory, pdTRUE, pdTRUE, portMAX_DELAY );
    xSemaphoreTake( sema_HistoryCompleted, portMAX_DELAY );
    oPressureHistoryPtr[IntDataPtr[1]] = ePtr[5];
    //oIAQ_HistoryPtr[IntDataPtr[1]] = ePtr[6];
    //oHumidityHistoryPtr[IntDataPtr[1]] = ePtr[4];
    //oTemperaturePtr[IntDataPtr[1]] = ePtr[3];
    StorageTriggerCount++; //triggered by the timer isr once a minute
    log_i( "storage trigger count %d", StorageTriggerCount );
    if ( StorageTriggerCount == maxStorageTriggerCount )
    {
      IntDataPtr[1]++;
          log_i( "IntDataPtr %d", IntDataPtr[1] );
      if ( IntDataPtr[1] == IntDataPtr[0] )
      {
        IntDataPtr[1] = 0;
      }
      StorageTriggerCount = 0;
    }
    xSemaphoreGive( sema_HistoryCompleted );
    // log_i( " high watermark %d",  uxTaskGetStackHighWaterMark( NULL ) );
  }
  vTaskDelete( NULL );
} //void fCollectHistory( void * parameter )
////
// Using a semaphore to protect the PSRAM from multiple tasks access the same PSRM locations
////
void fUpdateDisplay( void * parameter )
{
  // Color definitions
  // http://www.barth-dev.de/online/rgb565-color-picker/
  /*
    ColorPalettePtr[0] = 0x0000; //BLACK
    ColorPalettePtr[1] = 0x001F; //BLUE
    ColorPalettePtr[2] = 0xF800; //RED
    ColorPalettePtr[3] = 0x07E0; //GREEN
    ColorPalettePtr[4] = 0x07FF; //CYAN
    ColorPalettePtr[5] = 0xF81F; //MAGENTA
    ColorPalettePtr[6] = 0xFFE0; //YELLOW
    ColorPalettePtr[7] = 0xFFFF; //WHITE
    ColorPalettePtr[8] = 0xFFFFD8; //LIGHTYELLOW
    ColorPalettePtr[9] = 0xFF8040; //BROWN
  */
  for (;;)
  {
    xEventGroupWaitBits (eg, evtDoDisplay, pdTRUE, pdTRUE, portMAX_DELAY ); //
    vTaskDelay( 10 );
    tft.fillScreen( ColorPalettePtr[0] );
    tft.setTextColor( ColorPalettePtr[7] );
    tft.setCursor( 0, 0 );
    tft.print( "Inside"  );
    tft.setTextColor( ColorPalettePtr[3] );
    tft.setCursor( 0, 15 );
    tft.print( "Temp: " + String(ePtr[0]) + "C " + String((ePtr[0] * 9 / 5) + 32) + "F"  );
    tft.setCursor( 0, 25 );
    tft.print( "Humidity " + String(ePtr[2]) + "%" );
    //
    tft.setTextColor( ColorPalettePtr[7] );
    tft.setCursor( 0,  40 );
    tft.print( "Outside" );
    tft.setTextColor( ColorPalettePtr[6] );
    tft.setCursor( 0, 55 );
    tft.print( "Temperature: " + String(ePtr[3]) + "F" );
    tft.setTextColor( ColorPalettePtr[2] );
    tft.setCursor( 0, 65 );
    tft.print( "Humidity " + String(ePtr[4]) + "%" );
    tft.setCursor( 0, 75 );
    tft.setTextColor( ColorPalettePtr[1] );
    tft.print( "Pres. " + String(ePtr[5]) + "mmHg" );
    tft.setCursor( 0, 86 );
    //set the color of the value to be displayed
    if ( ePtr[6] < 51.0f )
    {
      tft.setTextColor( ColorPalettePtr[3] );
    }
    if ( (ePtr[6] >= 50.0f) && (ePtr[6] <= 100.0f) )
    {
      tft.setTextColor( ColorPalettePtr[6] );
    }
    if ( (ePtr[6] >= 100.0f) && (ePtr[6] <= 150.0f) )
    {
      tft.setTextColor( ColorPalettePtr[9] );
    }
    if ( (ePtr[6] >= 150.0f) && (ePtr[6] <= 200.0f) )
    {
      tft.setTextColor( ColorPalettePtr[2] );
    }
    if ( (ePtr[6] >= 200.00f) && (ePtr[6] <= 300.0f) )
    {
      tft.setTextColor( ColorPalettePtr[5] );
    }
    if ( (ePtr[6] > 300.0f) )
    {
      tft.setTextColor( ColorPalettePtr[7] );
    }
    tft.print( "AQ Index " + String(ePtr[6]) );
    tft.setTextColor( ColorPalettePtr[1] ); //set graph line color
    int rowRef = 110;
    int hRef = int( oPressureHistoryPtr[0] );
    int nextPoint = 2;
    int nextCol = 0;
    xSemaphoreTake( sema_HistoryCompleted, portMAX_DELAY );
    for (int i = 0; i < IntDataPtr[0]; i++)
    {
      int hDisplacement = hRef - int( oPressureHistoryPtr[i] ); // cell height displacement from base line
      tft.setCursor( nextCol , (rowRef + hDisplacement) );
      tft.print( "_" );
      nextCol += nextPoint;
    }
    tft.setCursor( (IntDataPtr[1] * nextPoint), (rowRef + 3) );
    tft.print( "I" );
    xSemaphoreGive( sema_HistoryCompleted );
    //     log_i( "fUpdateDisplay MEMORY WATERMARK %d", uxTaskGetStackHighWaterMark(NULL) );
  }
  vTaskDelete( NULL );
} // void fUpdateDisplay( void * parameter )
////
void DoTheBME280Thing( void *pvParameters )
{
  for (;;)
  {
    xEventGroupWaitBits (eg, evtDoTheBME280Thing, pdTRUE, pdTRUE, portMAX_DELAY ); //
    vTaskDelay( 2 );
    if ( !isnan(bme.readTemperature()) )
    {
      ePtr[0] = bme.readTemperature();
      ePtr[1] = bme.readPressure() / 133.3223684f; // mmHg
      ePtr[2] = bme.readHumidity();
      if ( MQTTclient.connected() ) // broker online, then send stuff
      {
        xSemaphoreTake( sema_MQTT_KeepAlive, portMAX_DELAY ); // do not send whiles keep alive is in progress and pause keep-alive during send
        MQTTclient.publish( topicInsideTemp, String(ePtr[0]).c_str(), true );
        vTaskDelay( 2 ); // gives the Raspberry Pi 4 time to receive the message and process
        MQTTclient.publish( topicInsideHumidity, String(ePtr[1]).c_str(), true ); // retained set to true
        vTaskDelay( 2 ); // no delay and RPi is still processing previous message
        MQTTclient.publish( topicInsidePressure, String(ePtr[2]).c_str(), true );
        xSemaphoreGive( sema_MQTT_KeepAlive );
      }
    }
    //log_i( "DoTheBME280Thing high watermark %d",  uxTaskGetStackHighWaterMark( NULL ) );
  }
  vTaskDelete ( NULL );
}
////
38 
    Impostant to not set vtaskDelay to less then 10. Errors begin to develop with the MQTT and network connection.
    makes the initial wifi/mqtt connection and works to keeps those connections open.
*/
void MQTTkeepalive( void *pvParameters )
{
  // setting must be set before a mqtt connection is made
  MQTTclient.setKeepAlive( 90 ); // setting keep alive to 90 seconds
  for (;;)
  {
    if ( (wifiClient.connected()) && (WiFi.status() == WL_CONNECTED) )
    {
      xSemaphoreTake( sema_MQTT_KeepAlive, portMAX_DELAY ); //
      MQTTclient.loop();
      xSemaphoreGive( sema_MQTT_KeepAlive );
    }
    else {
      log_i( "MQTT keep alive found MQTT status %s WiFi status %s", String(wifiClient.connected()), String(WiFi.status()) );
      if ( !(WiFi.status() == WL_CONNECTED) )
      {
        connectToWiFi();
      }
      connectToMQTT();
    }
    vTaskDelay( 250 );
  }
  vTaskDelete ( NULL );
}
////
void connectToMQTT()
{
  log_i( "connect to mqtt" );
  while ( !MQTTclient.connected() )
  {
    MQTTclient.connect( clientID, mqtt_username, mqtt_password );
    log_i( "connecting to MQTT" );
    vTaskDelay( 250 );
  }
  log_i("MQTT Connected");
  MQTTclient.setCallback( mqttCallback );
  MQTTclient.subscribe( mqtt_topic );
}
//
void connectToWiFi()
{
  log_i( "connect to wifi" );
  while ( WiFi.status() != WL_CONNECTED )
  {
    WiFi.disconnect();
    WiFi.begin( SSID, PASSWORD );
    log_i(" waiting on wifi connection" );
    vTaskDelay( 4000 );
  }
  log_i( "Connected to WiFi" );
  WiFi.macAddress(mac);
  log_i( "mac address %d.%d.%d.%d.%d", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]  );
  WiFi.onEvent( WiFiEvent );
  GetTheTime();
  printLocalTime();
}
////
void fparseMQTT( void *pvParameters )
{
  xSemaphoreGive ( sema_MQTT_Parser );
  for (;;)
  {
    xEventGroupWaitBits (eg, evtParseMQTT, pdTRUE, pdTRUE, portMAX_DELAY ); //
    xSemaphoreTake( sema_MQTT_Parser, portMAX_DELAY );
    if ( (String)str_eTopicPtr == topicOutsideTemperature )
    {
      ePtr[3] = String(strPayloadPtr).toFloat();
      // log_i( "oTemperature %f", ePtr[3] );
    }
    if ( (String)str_eTopicPtr == topicOutsideHumidity )
    {
      ePtr[4] = String(strPayloadPtr).toFloat();
    }
    if ( (String)str_eTopicPtr == topicAQIndex )
    {
      ePtr[6] = String(strPayloadPtr).toFloat();
    }
    if ( String(str_eTopicPtr) == topicStateLED )
    {
      if ( String(strPayloadPtr).toInt() )
      {
        REG_WRITE(GPIO_OUT_W1TC_REG, BIT0);
      } else {
        REG_WRITE(GPIO_OUT_W1TS_REG, BIT0);
      }
    }
    if ( String(str_eTopicPtr) == topicOutsidePressure ) // see note below
    {
      ePtr[5] = String(strPayloadPtr).toFloat();
      //log_i( "oPressure %f", ePtr[5] );
    }
    if ( (String)str_eTopicPtr == topic_hum_enable )
    {
      xSemaphoreTake( sema_TheHumidityThing, portMAX_DELAY );
      IntDataPtr[3] = String(strPayloadPtr).toInt();
      xSemaphoreGive( sema_TheHumidityThing );
    }
    if ( (String)str_eTopicPtr == topic_hum_setpoint )
    {
      xSemaphoreTake( sema_TheHumidityThing, portMAX_DELAY );
      IntDataPtr[2] = String(strPayloadPtr).toInt();
      xSemaphoreGive( sema_TheHumidityThing );
    }
    if ( (String)str_eTopicPtr == topic_SunLampOn )
    {
      xSemaphoreTake( sema_DoTheSunLampTHing, portMAX_DELAY );
      IntDataPtr[4] = String(strPayloadPtr).toInt();
      xSemaphoreGive( sema_DoTheSunLampTHing );
    }
    if ( (String)str_eTopicPtr == topic_SunLampEnable )
    {
      xSemaphoreTake( sema_DoTheSunLampTHing, portMAX_DELAY );
      IntDataPtr[5] = String(strPayloadPtr).toInt();
      xSemaphoreGive( sema_DoTheSunLampTHing );
    }
    // clear pointer locations
    for ( int i = 0; i < 21; i++ )
    {
      strPayloadPtr[i] = '\0';
      strPayloadPtr[i] = '\0';
    }
    xSemaphoreGive( sema_MQTT_Parser );
  }
} // void fparseMQTT( void *pvParameters )[/code
39 
/*
   frequent disconnects make for more realible data receiption when data changes
   Pressure is the last data item sents, mqttkeepalive will cause reconnection
*/
////
// Important to get as much code out of the callback for realible operations.
////
static void mqttCallback(char* topic, byte * payload, unsigned int length)
{
  xSemaphoreTake( sema_MQTT_Parser, portMAX_DELAY);
  str_eTopicPtr = topic + '\0';
  int i = 0;
  for ( i; i < length; i++) {
    strPayloadPtr[i] = ((char)payload[i]);
  }
  strPayloadPtr[i] = '\0';
  //log_i( "topic %s payload %s" ,str_eTopicPtr, strPayloadPtr );
  xSemaphoreGive ( sema_MQTT_Parser );
  xEventGroupSetBits( eg, evtParseMQTT ); // trigger tasks
} // void mqttCallback(char* topic, byte* payload, unsigned int length)
////
// great trouble shooting tool when uncommented
////
void WiFiEvent(WiFiEvent_t event)
{
  // log_i( "[WiFi-event] event: %d\n", event );
  switch (event) {
    //    case SYSTEM_EVENT_WIFI_READY:
    //      log_i("WiFi interface ready");
    //      break;
    //    case SYSTEM_EVENT_SCAN_DONE:
    //      log_i("Completed scan for access points");
    //      break;
    //    case SYSTEM_EVENT_STA_START:
    //      log_i("WiFi client started");
    //      break;
    //    case SYSTEM_EVENT_STA_STOP:
    //      log_i("WiFi clients stopped");
    //      break;
    case SYSTEM_EVENT_STA_CONNECTED:
      log_i("Connected to access point");
      break;
    case SYSTEM_EVENT_STA_DISCONNECTED:
      log_i("Disconnected from WiFi access point");
      break;
    //    case SYSTEM_EVENT_STA_AUTHMODE_CHANGE:
    //      log_i("Authentication mode of access point has changed");
    //      break;
    //    case SYSTEM_EVENT_STA_GOT_IP:
    //      log_i ("Obtained IP address: %s",  WiFi.localIP() );
    //      break;
    //    case SYSTEM_EVENT_STA_LOST_IP:
    //      log_i("Lost IP address and IP address is reset to 0");
    //      //      vTaskDelay( 5000 );
    //      //      ESP.restart();
    //      break;
    //    case SYSTEM_EVENT_STA_WPS_ER_SUCCESS:
    //      log_i("WiFi Protected Setup (WPS): succeeded in enrollee mode");
    //      break;
    //    case SYSTEM_EVENT_STA_WPS_ER_FAILED:
    //      log_i("WiFi Protected Setup (WPS): failed in enrollee mode");
    //      //      ESP.restart();
    //      break;
    //    case SYSTEM_EVENT_STA_WPS_ER_TIMEOUT:
    //      log_i("WiFi Protected Setup (WPS): timeout in enrollee mode");
    //      break;
    //    case SYSTEM_EVENT_STA_WPS_ER_PIN:
    //      log_i("WiFi Protected Setup (WPS): pin code in enrollee mode");
    //      break;
    //    case SYSTEM_EVENT_AP_START:
    //      log_i("WiFi access point started");
    //      break;
    //    case SYSTEM_EVENT_AP_STOP:
    //      log_i("WiFi access point  stopped");
    //      //      WiFi.mode( WIFI_OFF);
    //      //      esp_sleep_enable_timer_wakeup( 1000000 * 2 ); // 1 second times how many seconds wanted
    //      //      esp_deep_sleep_start();
    //      break;
    //    case SYSTEM_EVENT_AP_STACONNECTED:
    //      log_i("Client connected");
    //      break;
    case SYSTEM_EVENT_AP_STADISCONNECTED:
      log_i("WiFi client disconnected");
    //      break;
    //    case SYSTEM_EVENT_AP_STAIPASSIGNED:
    //      log_i("Assigned IP address to client");
    //      break;
    //    case SYSTEM_EVENT_AP_PROBEREQRECVED:
    //      log_i("Received probe request");
    //      break;
    //    case SYSTEM_EVENT_GOT_IP6:
    //      log_i("IPv6 is preferred");
    //      break;
    //    case SYSTEM_EVENT_ETH_GOT_IP:
    //      log_i("Obtained IP address");
    //      break;
    default: break;
  }
}
////
void loop() { }
40

Idahowalker:
Try.

void reconnect() {

mqttClient.setKeepAlive( 90 );
  // Loop until we're reconnected
  while (!mqttClient.connected()) {



I did not see where a callback is defined?

It's there up towards the top, but appears like it is not getting called.

Thanks for the examples. It will take some time to work through them.