Custom MQTT parameters using Wifi Manager

Hello, everyone.

On my Adafruit HUZZAH32 - ESP32 Feather I would like to configure MQTT connection to Adafruit IO platform using WiFi manager (web captive portal), but the problem is that MQTT connection, publish and subscribe functions ask for global definitions (#define) and these, as we all know, are non rewrittable.

I am using ConfigOnSwitchFS example by khoih-prog developer. For MQTT broker I am using Adafruit MQTT and their modified for ESP32 mqtt_2subs_esp8266 example.

Basically, I modified ConfigOnSwitchFS by adding more parameter fields (forms) like MQTT username, password etc. and I am able to save these new paramaters to a SPIFFS file, but they are either char or int variables and adafruit's MQTT publish, subscribe and other functions won't accept them. They only accept #define's, like this:

#define AIO_SERVER_KEY              "io.adafruit.com"
#define AIO_SERVERPORT_KEY       "1883"
#define AIO_USERNAME_KEY          "someone"
#define AIO_KEY_KEY                    "key_example_12345"

This won't be accepted in function calls, even though i successfully managed to update these variables through web server and SPIFFS:

char AIO_SERVER_KEY[20] = "io.adafruit.com";
int AIO_SERVERPORT_KEY = 1883; 
char AIO_USERNAME_KEY[20] = "someone";
char AIO_KEY_KEY[40] = "key_example_12345";

And this is how whole setup for connection to Adafruit IO looks like:

// Create an ESP32 WiFiClient class to connect to the MQTT server.
WiFiClient client;

// Setup the MQTT client class by passing in the WiFi client and MQTT server and login details.
Adafruit_MQTT_Client mqtt1(&client, AIO_SERVER_KEY, AIO_SERVERPORT_KEY, AIO_USERNAME_KEY, AIO_KEY_KEY);

// Feeds
Adafruit_MQTT_Subscribe BME280_onoff1 = Adafruit_MQTT_Subscribe(&mqtt1, AIO_USERNAME_KEY "/feeds/bme280_onoff");
Adafruit_MQTT_Publish Temperatura1 = Adafruit_MQTT_Publish(&mqtt1, AIO_USERNAME_KEY "/feeds/Temperatura");

This is in the source code (Adafruit_MQTT.h):

class Adafruit_MQTT_Publish {
 public:
  Adafruit_MQTT_Publish(Adafruit_MQTT *mqttserver, const char *feed, uint8_t qos = 0);

  bool publish(const char *s);
  bool publish(double f, uint8_t precision=2);  // Precision controls the minimum number of digits after decimal.
                                                // This might be ignored and a higher precision value sent.
  bool publish(int32_t i);
  bool publish(uint32_t i);
  bool publish(uint8_t *b, uint16_t bLen);


private:
  Adafruit_MQTT *mqtt;
  const char *topic;
  uint8_t qos;
};

class Adafruit_MQTT_Subscribe {
 public:
  Adafruit_MQTT_Subscribe(Adafruit_MQTT *mqttserver, const char *feedname, uint8_t q=0);

  void setCallback(SubscribeCallbackUInt32Type callb);
  void setCallback(SubscribeCallbackDoubleType callb);
  void setCallback(SubscribeCallbackBufferType callb);
  void setCallback(AdafruitIO_MQTT *io, SubscribeCallbackIOType callb);
  void removeCallback(void);

  const char *topic;
  uint8_t qos;

  uint8_t lastread[SUBSCRIPTIONDATALEN];
  // Number valid bytes in lastread. Limited to SUBSCRIPTIONDATALEN-1 to
  // ensure nul terminating lastread.
  uint16_t datalen;

  SubscribeCallbackUInt32Type callback_uint32t;
  SubscribeCallbackDoubleType callback_double;
  SubscribeCallbackBufferType callback_buffer;
  SubscribeCallbackIOType     callback_io;

  AdafruitIO_MQTT *io_mqtt;

 private:
  Adafruit_MQTT *mqtt;
};

Additional info requirements: I read somewere that the initial MQTT parameters are stored in a flash memory and must always be present, that's why Adafruit example uses only #defines. So now I must somehow point function calls to take arguments from newly written parameters in flash memory, I guess using pointers (not experienced enough to use them, though).

My MQTT setup functions need to be done globally, that is before setup() because these function will be used later in code. I first call WiFi manager by clicking on a button, then I enter custom paramteres in web portal, after that those paramteres get saved in SPIFFS .txt file and are applied immedeately after reset. But, as mentioned, I wasn't able to incorporate these new parameters to connect to Adafruit IO by MQTT.

Please, advise.

they are either char or int variables and adafruit's MQTT publish, subscribe and other functions won't accept them.

What errors do you get when you try to use values from SPIFFS and what data type are the variables ?

class Adafruit_MQTT_Subscribe
{
  public:
    Adafruit_MQTT_Subscribe(Adafruit_MQTT *mqttserver, const char *feedname, uint8_t q = 0);

Note how this function takes pointers as its first two parameters

I note that all of the #defined values in your example are string constants and none of them are ints

UKHeliBob:
What errors do you get when you try to use values from SPIFFS and what data type are the variables ?

When I try to use char and int variables instead of #defines, I get these combinations of errors:

error: expected primary-expression before '(' token

 Adafruit_MQTT_Subscribe BME280_onoff1 = Adafruit_MQTT_Subscribe(&mqtt1, AIO_USERNAME_KEY "/feeds/bme280_onoff");

                                                                ^

error: expected ')' before string constant

 Adafruit_MQTT_Subscribe BME280_onoff1 = Adafruit_MQTT_Subscribe(&mqtt1, AIO_USERNAME_KEY "/feeds/bme280_onoff");

                                                                                          ^



error: expected primary-expression before '(' token

 Adafruit_MQTT_Publish Temperatura1 = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME_KEY "/feeds/Temperatura");

                                                          ^

error: expected ')' before string constant

 Adafruit_MQTT_Publish Temperatura = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME_KEY "/feeds/Temperatura");
                                                                                  ^

Custom paramteres types saved in WiFi manager are the same as those declared in the sketch, so they get directly overwritten after ESP32 reset.

UKHeliBob:
I note that all of the #defined values in your example are string constants and none of them are ints

Hmm, I wasn't excatly aware what type of constants are #defines, but here is Adafruit_MQTT_Client class definition from cpp file:

class Adafruit_MQTT_Client : public Adafruit_MQTT {
 public:
  Adafruit_MQTT_Client(Client *client, const char *server, uint16_t port,
                       const char *cid, const char *user, const char *pass):
    Adafruit_MQTT(server, port, cid, user, pass),
    client(client)
  {}

  Adafruit_MQTT_Client(Client *client, const char *server, uint16_t port,
                       const char *user="", const char *pass=""):
    Adafruit_MQTT(server, port, user, pass),
    client(client)
  {}

  bool connectServer();
  bool disconnectServer();
  bool connected();
  uint16_t readPacket(uint8_t *buffer, uint16_t maxlen, int16_t timeout);
  bool sendPacket(uint8_t *buffer, uint16_t len);

 private:
  Client* client;
};

Server port is uint16_t integer.

UKHeliBob:
Note how this function takes pointers as its first two parameters

So is there a simple solution to apply custom MQTT parameters from flash or other memory? Why do these functions only accept #defines? Can we make them accept regular chars and ints or point them to those new paramteres saved in SPIFFS after transfering them through WiFi manager?

Unfortunately, I am not really skilled in pointers. Could you please elaborate a little bit more?

Why do these functions only accept #defines?

That is not what is happening

A #define simply says to the compiler pre-processor replace this with that before compiling
So

#define AIO_SERVER_KEY              "io.adafruit.com"

causes all instances of AIO_SERVER_KEY to be replaced by "io.adafruit.com" in the sketch just as if you had typed it in

If you are going to use variables instead of #defines then they must be of the correct type
You say that

char AIO_SERVER_KEY[20] = "io.adafruit.com";

is not accepted
Try

char * AIO_SERVER_KEY = "io.adafruit.com";

If that works then you will know that the variable holding the value read from SPIFFS needs to be a char pointer

NOTE that I have no experience of the MQTT library that you are using nor can I test it

Yes, I am trying to use variables instead of #defines because latter can no be changed/written to by WiFi manager or anyhow. User must be able to add his custom parameters to connect to Adafruit IO platform through MQTT connection. After making changes you suggested, I get new errors in addition to old ones:

ConfigOnSwitchFS_moj2:146:120: error: invalid conversion from 'int*' to 'uint16_t {aka short unsigned int}' [-fpermissive]

 Adafruit_MQTT_Client mqtt1(&client, AIO_SERVER_KEY, AIO_SERVERPORT_KEY, AIO_USERNAME_KEY, AIO_USERNAME_KEY, AIO_KEY_KEY);

                                                                                                                        ^

In file included from C:...\Arduino\ConfigOnSwitchFS_moj2\ConfigOnSwitchFS_moj2.ino:14:0:

C:\...\Arduino\libraries\Adafruit_MQTT_Library/Adafruit_MQTT_Client.h:38:3: note:   initializing argument 3 of 'Adafruit_MQTT_Client::Adafruit_MQTT_Client(Client*, const char*, uint16_t, const char*, const char*, const char*)'

   Adafruit_MQTT_Client(Client *client, const char *server, uint16_t port,

   ^

ConfigOnSwitchFS_moj2:149:64: error: expected primary-expression before '(' token

 Adafruit_MQTT_Subscribe BME280_onoff1 = Adafruit_MQTT_Subscribe(&mqtt1, AIO_USERNAME_KEY "/feeds/bme280_onoff");

                                                                ^

ConfigOnSwitchFS_moj2:149:90: error: expected ')' before string constant

 Adafruit_MQTT_Subscribe BME280_onoff1 = Adafruit_MQTT_Subscribe(&mqtt1, AIO_USERNAME_KEY "/feeds/bme280_onoff");

                                                                                          ^

ConfigOnSwitchFS_moj2:152:59: error: expected primary-expression before '(' token

 Adafruit_MQTT_Publish Temperatura1 = Adafruit_MQTT_Publish(&mqtt1, AIO_USERNAME_KEY "/feeds/Temperatura");

                                                           ^

ConfigOnSwitchFS_moj2:152:85: error: expected ')' before string constant

 Adafruit_MQTT_Publish Temperatura1 = Adafruit_MQTT_Publish(&mqtt1, AIO_USERNAME_KEY "/feeds/Temperatura");

                                                                                     ^

Let me just ask you this: Do I have to frist write to SPIFFS flash some default values to get the same result as if i was using #defines, or do I leave them just as they are? Because these client/publish/subscribe functions get called before setup() and loop(), and after entering new parameters through WiFi manager, they get called again but this time using those new entered parameters (hopefully).

It is going to be impossible to help without seeing your complete sketch but most of it is going to be irrelevant to the problem you are having

I suggest that you write a bare minimum sketch that uses #defines and that connects to the MQTT server and simply prints a message indicating that is has done so. Do that and post it here and we can then take it further, first by using variables then moving on to using values from SPIFFS

Doing it that way you are not trying to solve multiple problems at once

What is the objective of the project and why do you want to load values from SPIFFS ?

I expect that the Adafruit examples are using #define because those things are fixed. They know that they will be using their own MQTT server so why make it variable? Similarly, if you want to subscribe to a topic, it's going to have a fixed name, so it may as well be constant too.

I'm sure that there are obscure examples of a need for MQTT code to switch between servers and topics on the fly, but the example code isn't intended to cater to that. Hence their choice of #define.

You can certainly use variables to hold that data, you just have to make sure that they're the appropriate type or cast to one.

This is the first part of whole code (I have to split it because of max message lenght)

#include <Arduino.h> // for button
#include <OneButton.h> // for button

#include <FS.h>
#include <ArduinoJson.h>
#include "SPIFFS.h"
#include <esp_wifi.h>
#include <WiFi.h>
#include <WiFiClient.h>

#include "Adafruit_MQTT.h"
#include "Adafruit_MQTT_Client.h"

#define ESP_getChipId()   ((uint32_t)ESP.getEfuseMac()) //WiFi Manager
#define FileFS      SPIFFS

// SSID and PW for Config Portal
String ssid = "ESP_" + String(ESP_getChipId(), HEX);
const char* password = "your_password";

// SSID and PW for your Router
String Router_SSID;
String Router_Pass;

// For displaying Available Pages in Information Page of Config Portal
// Must be placed before #include <ESP_WiFiManager.h>
#define USE_AVAILABLE_PAGES     true
#include <ESP_WiFiManager.h>              //https://github.com/khoih-prog/ESP_WiFiManager

const int BUTTON_PIN = 13;
const int RED_LED = 26;
const int BLUE_LED = 25;
const int LED_ON = HIGH;
const int LED_OFF = LOW;

uint32_t timer = millis();
int some_number = 5;

const char* CONFIG_FILE = "/ConfigSW.json";

// Indicates whether ESP has WiFi credentials saved from previous session
bool initialConfig = true; //default false

// Default configuration values for Adafruit IO MQTT
// This actually works
#define AIO_SERVER              "io.adafruit.com"
#define AIO_SERVERPORT          1883 //1883, or 8883 for SSL
#define AIO_USERNAME            "private" //Adafruit IO
#define AIO_KEY                 "private"

// Labels for custom parameters in WiFi manager
#define AIO_SERVER_Label             "AIO_SERVER_Label"
#define AIO_SERVERPORT_Label         "AIO_SERVERPORT_Label"
#define AIO_USERNAME_Label           "AIO_USERNAME_Label"
#define AIO_KEY_Label                "AIO_KEY_Label"

// Variables to save custom parameters to...
// I would like to use these instead of #defines
char custom_AIO_SERVER[20];
int custom_AIO_SERVERPORT;
char custom_AIO_USERNAME[20];
char custom_AIO_KEY[40];

// Function Prototypes

bool readConfigFile();
bool writeConfigFile();

IPAddress stationIP   = IPAddress(192, 168, 1, 114); //*,*,2,*
IPAddress gatewayIP   = IPAddress(192, 168, 1, 1); //*,*,2,*
IPAddress netMask     = IPAddress(255, 255, 255, 0);
IPAddress dns1IP      = gatewayIP;
IPAddress dns2IP      = IPAddress(8, 8, 8, 8);

void heartBeatPrint(void)
{
  static int num = 1;

  if (WiFi.status() == WL_CONNECTED)
    Serial.print("H");        // H means connected to WiFi
  else
    Serial.print("F");        // F means not connected to WiFi

  if (num == 80)
  {
    Serial.println();
    num = 1;
  }
  else if (num++ % 10 == 0)
  {
    Serial.print(" ");
  }
}

void check_status()
{
  static ulong checkstatus_timeout = 0;

#define HEARTBEAT_INTERVAL    10000L
  // Print hearbeat every HEARTBEAT_INTERVAL (10) seconds.
  if ((millis() > checkstatus_timeout) || (checkstatus_timeout == 0))
  {
    heartBeatPrint();
    checkstatus_timeout = millis() + HEARTBEAT_INTERVAL;
  }
}

//Button config
OneButton btn = OneButton(
                  BUTTON_PIN,  // Input pin for the button
                  true,        // Button is active LOW
                  true         // Enable internal pull-up resistor
                );

// Create an ESP32 WiFiClient class to connect to the MQTT server
WiFiClient client;

// Setup the MQTT client class by passing in the WiFi client and MQTT server and login details
Adafruit_MQTT_Client mqtt(&client, AIO_SERVER, AIO_SERVERPORT, AIO_USERNAME, AIO_KEY);

// Feeds
Adafruit_MQTT_Publish Temperatura = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME "/feeds/Temperatura");

// Setup function
void setup()
{

  // Put your setup code here, to run once
  Serial.begin(115200);
  Serial.println("\nStarting");

  btn.attachClick(handleClick);
  btn.attachDoubleClick(handleDoubleClick);
  btn.attachLongPressStop(handleLongPressStop);

  // Initialize the LED digital pin as an output.
  pinMode(BLUE_LED, OUTPUT);

  // Mount the filesystem
  bool result = FileFS.begin();
  Serial.println("SPIFFS opened: " + result);

  if (!readConfigFile())
  {
    Serial.println("Failed to read configuration file, using default values");
  }

  unsigned long startedAt = millis();

  //Here starts the WiFi Manager initialization
  //Local intialization. Once its business is done, there is no need to keep it around
  ESP_WiFiManager ESP_wifiManager("ConfigOnSwichFS");

  ESP_wifiManager.setMinimumSignalQuality(-1);
  // Set static IP, Gateway, Subnetmask, DNS1 and DNS2. New in v1.0.5
  ESP_wifiManager.setSTAStaticIPConfig(stationIP, gatewayIP, netMask, dns1IP, dns2IP);

  // We can't use WiFi.SSID() in ESP32as it's only valid after connected.
  // SSID and Password stored in ESP32 wifi_ap_record_t and wifi_config_t are also cleared in reboot
  // Have to create a new function to store in EEPROM/SPIFFS for this purpose
  Router_SSID = ESP_wifiManager.WiFi_SSID();
  Router_Pass = ESP_wifiManager.WiFi_Pass();

  //Remove this line if you do not want to see WiFi password printed
  Serial.println("Stored: SSID = " + Router_SSID + ", Pass = " + Router_Pass);

  // SSID to uppercase
  ssid.toUpperCase();

  if (Router_SSID == "")
  {
    Serial.println("We haven't got any access point credentials, so get them now");

    digitalWrite(BLUE_LED, LED_ON); // Turn led on as we are in configuration mode.

    //it starts an access point
    //and goes into a blocking loop awaiting configuration
    if (!ESP_wifiManager.startConfigPortal((const char *) ssid.c_str(), password))
      Serial.println("Not connected to WiFi but continuing anyway.");
    else
      Serial.println("WiFi connected...yeey :)");
  }

  digitalWrite(BLUE_LED, LED_OFF); // Turn led off as we are not in configuration mode.

#define WIFI_CONNECT_TIMEOUT        30000L
#define WHILE_LOOP_DELAY            200L
#define WHILE_LOOP_STEPS            (WIFI_CONNECT_TIMEOUT / ( 3 * WHILE_LOOP_DELAY ))

  startedAt = millis();

  while ( (WiFi.status() != WL_CONNECTED) && (millis() - startedAt < WIFI_CONNECT_TIMEOUT ) )
  {
    WiFi.mode(WIFI_STA);
    WiFi.persistent (true);

    // We start by connecting to a WiFi network
    Serial.print("Connecting to ");
    Serial.println(Router_SSID);

    WiFi.config(stationIP, gatewayIP, netMask);
    //WiFi.config(stationIP, gatewayIP, netMask, dns1IP, dns2IP);

    WiFi.begin(Router_SSID.c_str(), Router_Pass.c_str());

    int i = 0;
    while ((!WiFi.status() || WiFi.status() >= WL_DISCONNECTED) && i++ < WHILE_LOOP_STEPS)
    {
      delay(WHILE_LOOP_DELAY);
    }
  }

  Serial.print("After waiting ");
  Serial.print((millis() - startedAt) / 1000);
  Serial.print(" secs more in setup(), connection result is ");

  if (WiFi.status() == WL_CONNECTED)
  {
    Serial.print("connected. Local IP: ");
    Serial.println(WiFi.localIP());
  }
  else
    Serial.println(ESP_wifiManager.getStatus(WiFi.status()));
}

// Loop function

void loop()
{
  // checking button state all the time
  btn.tick();

  // this is just for checking if we are connected to WiFi
  check_status();

  // here we do publish to Adafruit IO platform every 10sec
  // we start by adjusting the time
  if (timer > millis()) {
    timer = millis();
  }

  // every 10 secons we upload value of some_number variable which is 5
  if (millis() - timer > 10000) {
    MQTT_connect_wifi();

    if (! Temperatura.publish(some_number)) {
      Serial.println(F("Failed to send value to Temperatura feed!"));
    }
    else {
      Serial.println(F("Value to Temperatura feed sucessfully sent!"));
    }
    
    timer = millis(); // reset the timer
  }
}

This is the second part of the whole code.

//event handler functions for button
static void handleClick() {
  Serial.println("Button clicked!");
  wifi_manager();
}

static void handleDoubleClick() {
  Serial.println("Button double clicked!");
}

static void handleLongPressStop() {
  Serial.println("Button pressed for long gitme and then released!");
  newConfigData();
}

void wifi_manager() {

  Serial.println("\nConfiguration portal requested.");
  digitalWrite(BLUE_LED, LED_ON); // turn the LED on by making the voltage LOW to tell us we are in configuration mode.

  //Local intialization. Once its business is done, there is no need to keep it around
  ESP_WiFiManager ESP_wifiManager;

  //Check if there is stored WiFi router/password credentials.
  //If not found, device will remain in configuration mode until switched off via webserver.
  Serial.print("Opening configuration portal. ");
  Router_SSID = ESP_wifiManager.WiFi_SSID();
  if (Router_SSID != "")
  {
    ESP_wifiManager.setConfigPortalTimeout(60); //If no access point name has been previously entered disable timeout.
    Serial.println("Got stored Credentials. Timeout 60s");
  }
  else
    Serial.println("No stored Credentials. No timeout");

  //Local intialization. Once its business is done, there is no need to keep it around

  // Extra parameters to be configured
  // After connecting, parameter.getValue() will get you the configured value
  // Format: <ID> <Placeholder text> <default value> <length> <custom HTML> <label placement>
  // (*** we are not using <custom HTML> and <label placement> ***)

  // AIO_SERVER
  ESP_WMParameter AIO_SERVER_FIELD(AIO_SERVER_Label, "AIO SERVER", custom_AIO_SERVER, 20);

  // AIO_SERVERPORT (because it is int, it needs to be converted to string)
  char convertedValue[5];
  sprintf(convertedValue, "%d", custom_AIO_SERVERPORT);
  ESP_WMParameter AIO_SERVERPORT_FIELD(AIO_SERVERPORT_Label, "AIO SERVER PORT", convertedValue, 5);

  // AIO_USERNAME
  ESP_WMParameter AIO_USERNAME_FIELD(AIO_USERNAME_Label, "AIO USERNAME", custom_AIO_USERNAME, 20);

  // AIO_KEY
  ESP_WMParameter AIO_KEY_FIELD(AIO_KEY_Label, "AIO KEY", custom_AIO_KEY, 20);

  // Just a quick hint
  ESP_WMParameter hint_text("<small>*If you want to connect to already connected AP, leave SSID and password fields empty</small>");

  // add all parameters here
  // order of adding is important
  ESP_wifiManager.addParameter(&hint_text);
  ESP_wifiManager.addParameter(&AIO_SERVER_FIELD);
  ESP_wifiManager.addParameter(&AIO_SERVERPORT_FIELD);
  ESP_wifiManager.addParameter(&AIO_USERNAME_FIELD);
  ESP_wifiManager.addParameter(&AIO_KEY_FIELD);

  // Sets timeout in seconds until configuration portal gets turned off.
  // If not specified device will remain in configuration mode until
  // switched off via webserver or device is restarted.
  // ESP_wifiManager.setConfigPortalTimeout(600);

  // It starts an access point
  // and goes into a blocking loop awaiting configuration.
  // Once the user leaves the portal with the exit button
  // processing will continue
  if (!ESP_wifiManager.startConfigPortal((const char *) ssid.c_str(), password))
  {
    Serial.println("Not connected to WiFi but continuing anyway.");
  }
  else
  {
    // If you get here you have connected to the WiFi
    Serial.println("Connected...yeey :)");
  }

  // Getting posted form values and overriding local variables parameters
  // Config file is written regardless the connection state
  strcpy(custom_AIO_SERVER, AIO_SERVER_FIELD.getValue());
  custom_AIO_SERVERPORT = atoi(AIO_SERVERPORT_FIELD.getValue());
  strcpy(custom_AIO_USERNAME, AIO_USERNAME_FIELD.getValue());
  strcpy(custom_AIO_KEY, AIO_KEY_FIELD.getValue());

  // Writing JSON config file to flash for next boot
  writeConfigFile();

  digitalWrite(BLUE_LED, LED_OFF); // Turn LED off as we are not in configuration mode.
}

bool readConfigFile() {

  // this opens the config file in read-mode
  File f = FileFS.open(CONFIG_FILE, "r");

  if (!f)
  {
    Serial.println("Configuration file not found");
    return false;
  }
  else
  {
    // we could open the file
    size_t size = f.size();
    // Allocate a buffer to store contents of the file.
    std::unique_ptr<char[]> buf(new char[size + 1]);

    // Read and store file contents in buf
    f.readBytes(buf.get(), size);
    // Closing file
    f.close();
    // Using dynamic JSON buffer which is not the recommended memory model, but anyway
    // See https://github.com/bblanchon/ArduinoJson/wiki/Memory%20model

    DynamicJsonDocument json(1024);
    auto deserializeError = deserializeJson(json, buf.get());
    if ( deserializeError )
    {
      Serial.println("JSON parseObject() failed");
      return false;
    }
    serializeJson(json, Serial);

    // Parse all config file parameters, override
    // local config variables with parsed values
    if (json.containsKey(AIO_SERVER_Label))
    {
      strcpy(custom_AIO_SERVER, json[AIO_SERVER_Label]);
    }

    if (json.containsKey(AIO_SERVERPORT_Label))
    {
      custom_AIO_SERVERPORT = json[AIO_SERVERPORT_Label];
    }

    if (json.containsKey(AIO_USERNAME_Label))
    {
      strcpy(custom_AIO_USERNAME, json[AIO_USERNAME_Label]);
    }

    if (json.containsKey(AIO_KEY_Label))
    {
      strcpy(custom_AIO_KEY, json[AIO_KEY_Label]);
    }
  }
  Serial.println("\nConfig file was successfully parsed");
  return true;
}

bool writeConfigFile() {

  Serial.println("Saving config file");

  DynamicJsonDocument json(1024);

  // JSONify local configuration parameters
  json[AIO_SERVER_Label] = custom_AIO_SERVER;
  json[AIO_SERVERPORT_Label] = custom_AIO_SERVERPORT;
  json[AIO_USERNAME_Label] = custom_AIO_USERNAME;
  json[AIO_KEY_Label] = custom_AIO_KEY;

  // Open file for writing
  File f = FileFS.open(CONFIG_FILE, "w");

  if (!f)
  {
    Serial.println("Failed to open config file for writing");
    return false;
  }

  serializeJsonPretty(json, Serial);
  // Write data to file and close it
  serializeJson(json, f);

  f.close();

  Serial.println("\nConfig file was successfully saved");
  return true;
}

// this function is just to display newly saved data,
// it is not necessary though, because data is displayed
// after WiFi manager resets ESP32
void newConfigData() {
  Serial.println();
  Serial.print("custom_AIO_SERVER: "); Serial.println(custom_AIO_SERVER);
  Serial.print("custom_SERVERPORT: "); Serial.println(custom_AIO_SERVERPORT);
  Serial.print("custom_USERNAME_KEY: "); Serial.println(custom_AIO_USERNAME);
  Serial.print("custom_KEY: "); Serial.println(custom_AIO_KEY);
  Serial.println();
}

void MQTT_connect_wifi() {

  int8_t ret;

  /* Če smo že povezavni, se ustavi. */
  if (mqtt.connected()) {
    return;
  }

  Serial.println("Connecting to WiFi MQTT (3 attempts)...");

  uint8_t attempt = 3;
  while ((ret = mqtt.connect()) != 0) { // connect will return 0 for connected
    Serial.println(mqtt.connectErrorString(ret));
    Serial.println("Another attemtpt to connect to MQTT in 2 seconds...");
    mqtt.disconnect();
    delay(2000);  // wait 2 seconds
    attempt--;
    if (attempt == 0) {
      Serial.println("WiFi MQTT connection failed. Continuing with program...");
      return;
    }
  }
  Serial.println("WiFi MQTT connection successful!");
}

And then here is serial monitor result:

15:48:51.695 -> Stored: SSID = ITK Home, Pass = *private*
15:48:51.695 -> Connecting to ITK Home
15:48:52.107 -> After waiting 0 secs more in setup(), connection result is connected. Local IP: 192.168.1.114
15:48:52.107 -> HButton clicked!
15:48:56.479 -> 
15:48:56.479 -> Configuration portal requested.
15:48:56.479 -> Opening configuration portal. Got stored Credentials. Timeout 60s
15:49:15.243 -> dhcps: send_offer>>udp_sendto result 0
15:49:39.750 -> Connected...yeey :)
15:49:39.750 -> Saving config file
15:49:39.750 -> {
15:49:39.750 ->   "AIO_SERVER_Label": "my_new_aio_server",
15:49:39.750 ->   "AIO_SERVERPORT_Label": 1234,
15:49:39.750 ->   "AIO_USERNAME_Label": "my_new_username",
15:49:39.750 ->   "AIO_KEY_Label": "my_new_key_12345"
15:49:39.750 -> }
15:49:39.750 -> Config file was successfully saved
15:49:39.750 -> HConnecting to WiFi MQTT (3 attempts)...
15:49:40.194 -> WiFi MQTT connection successful!
15:49:40.194 -> Value to Temperatura feed sucessfully sent!
15:49:49.747 -> HValue to Temperatura feed sucessfully sent!

I also attached an image of WiFi configuration manager just to make whole thing clearer.

Program works like this: we turn on ESP32, it goes to while() loop, waits for button click. When we click the button, we call WiFi manager and there we configure some parameters. After clicking "save" button, ESP32 restarts and saves those parameters which MQTT function(s) should then accept and ESP32 should start sending data.

wildbill:
I expect that the Adafruit examples are using #define because those things are fixed. They know that they will be using their own MQTT server so why make it variable? Similarly, if you want to subscribe to a topic, it's going to have a fixed name, so it may as well be constant too.

What I would like to do is make weather station which is predetermined to send data to Adafruit IO platform using MQTT protocol, but also to be able to configure every configuration parameter through WiFi manager. That includes changing user name, key, feeds and subscriptions, time delay of sending data and so on...

wildbill:
I'm sure that there are obscure examples of a need for MQTT code to switch between servers and topics on the fly, but the example code isn't intended to cater to that. Hence their choice of #define.

Various users have various needs. My project requires this kind of flexibility and if it won't work with Adafruit MQTT, then I will be forced to use another broker or platform.

wildbill:
You can certainly use variables to hold that data, you just have to make sure that they're the appropriate type or cast to one.

Based on what I have posted now and before, could you please help me integrate those variables for my purpose?

Well, looking at the library, it's more tricky than I thought and now I see why their examples work that way. The MQTT objects are global, so there's no opportunity to call their constructors with variables.

What I suspect you would need is to create local (possibly static) instances instead, which would be plausible to create using c strings as parameters. I'll take a closer look when I'm near a computer.

I've searched whole internet in hopes of finding a solution to this problem. "adafruit mqtt custom parameters" and "adafruit mqtt dynamic configuration" yielded little to no results. There is a topic on this forum though, where one managed to update feed name, but that wasn't in the #defines as username and password are. Hope J-M-L will see my message and check this topic since he had some suggestions on that beforementioned topic.

Also to add, if we exclude unnecessary code and demands in this topic (my project), it all gets down to "non-flexibility" of Adafruit MQTT library to dynamic configuration and I should probably make seperate topic since I "clogged" this one with extra unhelpful info.

wildbill:
I'll take a closer look when I'm near a computer.

I will wait for wildbill's response and also a permission for above mentioned idea for a new topic, or any other proposal. Since this is not looking promising, I am seriously considering to switching to other MQTT libraries or platforms like Thingspeak. However, I do like Adafruit IO dashboards and features and wouldn't like to lose them, so I'll wait a little bit longer and hope for the best.

It looks plausible, although I can't prove it because I can't get the ESP board data to load.

I took the Adafruit example and tweaked it so that the onoffbutton subscription is a NULL pointer initially. Then in Setup, I created a static subscription object and pointed the onoffbutton pointer to it. I had to tweak the usage a bit in the rest of the code to deal with the fact that the pointer has to be dereferenced.

The reason for these shenanigans is that I was able to pass a variable for the username when constructing the subscription object. In my example, that's just a string literal, but you could use some other means to get that data from config first.

You would need to follow the same pattern for the client and publish objects.

Here's the code:

/***************************************************
  Adafruit MQTT Library ESP8266 Example
  Must use ESP8266 Arduino from:
    https://github.com/esp8266/Arduino
  Works great with Adafruit's Huzzah ESP board & Feather
  ----> https://www.adafruit.com/product/2471
  ----> https://www.adafruit.com/products/2821
  Adafruit invests time and resources providing this open source code,
  please support Adafruit and open-source hardware by purchasing
  products from Adafruit!
  Written by Tony DiCola for Adafruit Industries.
  MIT license, all text above must be included in any redistribution
 ****************************************************/
//#include <ESP8266WiFi.h>
#include "Adafruit_MQTT.h"
#include "Adafruit_MQTT_Client.h"

/************************* WiFi Access Point *********************************/

#define WLAN_SSID       "...your SSID..."
#define WLAN_PASS       "...your password..."

/************************* Adafruit.io Setup *********************************/

#define AIO_SERVER      "io.adafruit.com"
#define AIO_SERVERPORT  1883                   // use 8883 for SSL
#define AIO_USERNAME    "...your AIO username (see https://accounts.adafruit.com)..."
#define AIO_KEY         "...your AIO key..."

/************ Global State (you don't need to change this!) ******************/

// Create an ESP8266 WiFiClient class to connect to the MQTT server.
//WiFiClient client;
// or... use WiFiFlientSecure for SSL
//WiFiClientSecure client;

// Setup the MQTT client class by passing in the WiFi client and MQTT server and login details.
Adafruit_MQTT_Client mqtt(NULL, AIO_SERVER, AIO_SERVERPORT, AIO_USERNAME, AIO_KEY);

/****************************** Feeds ***************************************/

// Setup a feed called 'photocell' for publishing.
// Notice MQTT paths for AIO follow the form: <username>/feeds/<feedname>
Adafruit_MQTT_Publish photocell = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME "/feeds/photocell");

// Setup a feed called 'onoff' for subscribing to changes.
Adafruit_MQTT_Subscribe *onoffbutton=NULL;
//= Adafruit_MQTT_Subscribe(&mqtt, AIO_USERNAME "/feeds/onoff");

/*************************** Sketch Code ************************************/

// Bug workaround for Arduino 1.6.6, it seems to need a function declaration
// for some reason (only affects ESP8266, likely an arduino-builder bug).
void MQTT_connect();

void setup()
{
  static char MyUserName[]="username"; // Get this from your config rather than declaring a fixed string
  Serial.begin(115200);
  delay(10);
  Serial.println(F("Adafruit MQTT demo"));
  Adafruit_MQTT_Subscribe ooButton(&mqtt,MyUserName);
  onoffbutton=&ooButton;
  // Connect to WiFi access point.
  Serial.println(); Serial.println();
  Serial.print("Connecting to ");
  Serial.println(WLAN_SSID);
//  WiFi.begin(WLAN_SSID, WLAN_PASS);
//  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".");
  }
  Serial.println();
  Serial.println("WiFi connected");
  Serial.println("IP address: "); 
  //Serial.println(WiFi.localIP());
  // Setup MQTT subscription for onoff feed.
  mqtt.subscribe(onoffbutton);
}

uint32_t x = 0;

void loop()
{
  // Ensure the connection to the MQTT server is alive (this will make the first
  // connection and automatically reconnect when disconnected).  See the MQTT_connect
  // function definition further below.
  MQTT_connect();
  // this is our 'wait for incoming subscription packets' busy subloop
  // try to spend your time here
  Adafruit_MQTT_Subscribe *subscription;
  while ((subscription = mqtt.readSubscription(5000)))
  {
    if (subscription == onoffbutton)
    {
      Serial.print(F("Got: "));
      Serial.println((char *)onoffbutton->lastread);
    }
  }
  // Now we can publish stuff!
  Serial.print(F("\nSending photocell val "));
  Serial.print(x);
  Serial.print("...");
  if (! photocell.publish(x++))
  {
    Serial.println(F("Failed"));
  }
  else
  {
    Serial.println(F("OK!"));
  }
  // ping the server to keep the mqtt connection alive
  // NOT required if you are publishing once every KEEPALIVE seconds
  /*
    if(! mqtt.ping()) {
    mqtt.disconnect();
    }
  */
}

// Function to connect and reconnect as necessary to the MQTT server.
// Should be called in the loop function and it will take care if connecting.
void MQTT_connect()
{
  int8_t ret;
  // Stop if already connected.
  if (mqtt.connected())
  {
    return;
  }
  Serial.print("Connecting to MQTT... ");
  uint8_t retries = 3;
  while ((ret = mqtt.connect()) != 0)   // connect will return 0 for connected
  {
    Serial.println(mqtt.connectErrorString(ret));
    Serial.println("Retrying MQTT connection in 5 seconds...");
    mqtt.disconnect();
    delay(5000);  // wait 5 seconds
    retries--;
    if (retries == 0)
    {
      // basically die and wait for WDT to reset me
      while (1);
    }
  }
  Serial.println("MQTT Connected!");
}

I had to comment out the wifi library to get it to compile. Obviously, you won't.

wildbill, I followed you example but run into some problems. First, there was that same error as before:

C:\Users\...\ConfigOnSwitchFS_moj4.ino: In function 'void setup()':

ConfigOnSwitchFS_moj4:136:61: error: expected ')' before string constant

   Adafruit_MQTT_Publish temp_sub(&mqtt, custom_AIO_USERNAME "/feeds/Temperatura");

                                                             ^

So I used the strcpy() and strcat() functions to put some things together and got rid of that error. Here is all the relevant code, dots represent non-important missing code:

int some_number = 5;
.
.
.
char custom_AIO_SERVER[20];
int custom_AIO_SERVERPORT;
char custom_AIO_USERNAME[20];
char custom_AIO_KEY[40];

char username_and_feed1[50];
char username_and_feed2[50];
.
.
.
.
WiFiClient client;

Adafruit_MQTT_Client mqtt(&client, custom_AIO_SERVER, custom_AIO_SERVERPORT, custom_AIO_USERNAME, custom_AIO_KEY);

Adafruit_MQTT_Publish *Temperatura = NULL;

Adafruit_MQTT_Subscribe *BME280_onoff = NULL;

void setup()
{ 
  strcpy(username_and_feed1,custom_AIO_USERNAME);
  strcat(username_and_feed1,"/feeds/Temperatura");
  
  Adafruit_MQTT_Publish temp_sub(&mqtt, username_and_feed1);
  Temperatura = &temp_sub;
  
  strcpy(username_and_feed2,custom_AIO_USERNAME);
  strcat(username_and_feed2,"/feeds/bme280_onoff");
  
  Adafruit_MQTT_Subscribe BME_Button(&mqtt, username_and_feed2);
  BME280_onoff = &BME_Button;
  
  mqtt.subscribe(&BME_Button);
.
.
.
}

void loop()
{
    if (! temp_sub.publish(some_number)) {
      Serial.println(F("Failed to send value to Temperatura feed!"));
    }
    else {
      Serial.println(F("Value to Temperatura feed sucessfully sent!"));
    }
}

...but then I run into next error:

C:\Users\...\ConfigOnSwitchFS_moj4.ino: In function 'void loop()':

ConfigOnSwitchFS_moj4:267:11: error: 'temp_sub' was not declared in this scope

     if (! temp_sub.publish(some_number)) {

           ^

Looks like client object accepted custom variables, while subscribe and publish had to be done your way. It basically works, but now we need to make it global so I can use subscribe and publish in other functions too. Could you please do your magic again? :slight_smile:

Again, on a tablet for now. However, you need to use temperatura-> not temp_sub. Temp_sub is local to setup; it needs to be static too or the pointer will be pointing to garbage.

wildbill:
Again, on a tablet for now. However, you need to use temperatura-> not temp_sub.

That did the trick!

wildbill:
it needs to be static too or the pointer will be pointing to garbage.

Sorry, I didn't quite understand what needs to bo declared as static and how excatly do you do it? If I make Temperatura static, it throws an error:

error: 'Temperatura' does not name a type

   static Temperatura = &temp_sub;

          ^

Anyway, after doing the "->" trick, my program compiled but then there was another problem - it won't connect to MQTT. So I'm thinking maybe it is because of lack of static keyword, or maybe it because I didn't follow the same pattern of client object constructing as I did with subscribe and publish. So I tried fixing the latter by making these changes:

WiFiClient client;

Adafruit_MQTT_Client *mqtt = NULL;

Adafruit_MQTT_Publish *Temperatura = NULL;

Adafruit_MQTT_Subscribe *BME280_onoff = NULL;

void setup()
{ 
  Adafruit_MQTT_Client mqtt_connection(&client, custom_AIO_SERVER, custom_AIO_SERVERPORT, custom_AIO_USERNAME, custom_AIO_KEY);
  mqtt = &mqtt_connection;
  
  strcpy(username_and_feed1,custom_AIO_USERNAME);
  strcat(username_and_feed1,"/feeds/Temperatura");
  
  Adafruit_MQTT_Publish temp_sub(&mqtt_connection, username_and_feed1);
  Temperatura = &temp_sub;
  
  strcpy(username_and_feed2,custom_AIO_USERNAME);
  strcat(username_and_feed2,"/feeds/bme280_onoff");
  
  Adafruit_MQTT_Subscribe BME_Button(&mqtt_connection, username_and_feed2);
  BME280_onoff = &BME_Button;
  
  mqtt_connection.subscribe(&BME_Button);
.
.
.
}
void loop()
{
.
.
.
  // every 10 secons we upload value of some_number variable which is 5
  if (millis() - timer > 10000) {
    MQTT_connect_wifi();

    if (! Temperatura->publish(some_number)) {
      Serial.println(F("Failed to send value to Temperatura feed!"));
    }
    else {
      Serial.println(F("Value to Temperatura feed sucessfully sent!"));
    }
  }
}
.
.
.
//we also make proper (hopefully) changes in this function also
void MQTT_connect_wifi() {

  int8_t ret;

  if (mqtt->connected()) {
    return;
  }

  Serial.println("Connecting to WiFi MQTT (3 attempts)...");

  uint8_t attempt = 3;
  while ((ret = mqtt->connect()) != 0) { // connect will return 0 for connected
    Serial.println(mqtt->connectErrorString(ret));
    Serial.println("Another attemtpt to connect to MQTT in 2 seconds...");
    mqtt->disconnect();
    delay(2000);  // wait 2 seconds
    attempt--;
    if (attempt == 0) {
      Serial.println("WiFi MQTT connection failed. Continuing with program...");
      return;
    }
  }
  Serial.println("WiFi MQTT connection successful!");
}

This code compiled successfully, however it chrashes ESP32 as soon as it gets to MQTT_connect_wifi() in a loop().

11:41:38.710 -> Starting
11:41:38.812 -> PIFFS opened: 
11:41:38.812 -> {"AIO_SERVER_Label":"io.adafruit.com","AIO_SERVERPORT_Label":1883,"AIO_USERNAME_Label":" *private* ","AIO_KEY_Label":" *private* "}
11:41:38.812 -> Config file was successfully parsed
11:41:38.920 -> Stored: SSID = ITK Home, Pass = *private*
11:41:38.920 -> Connecting to ITK Home
11:41:39.125 -> After waiting 0 secs more in setup(), connection result is connected. Local IP: 192.168.1.114
11:41:39.125 -> HGuru Meditation Error: Core  1 panic'ed (InstrFetchProhibited). Exception was unhandled.
11:41:48.724 -> Core 1 register dump:
11:41:48.724 -> PC      : 0x310813b1  PS      : 0x00060230  A0      : 0x800d4ceb  A1      : 0x3ffb1f70  
11:41:48.724 -> A2      : 0x310813b1  A3      : 0x3ffc1068  A4      : 0x80088d90  A5      : 0x3ffb1e90  
11:41:48.724 -> A6      : 0x00000000  A7      : 0x3ffb0060  A8      : 0x800d4c64  A9      : 0x3ffb1f50  
11:41:48.724 -> A10     : 0x3ffb1e90  A11     : 0x00000000  A12     : 0x00002711  A13     : 0x000000c0  
11:41:48.724 -> A14     : 0x00000000  A15     : 0x3ffb0060  SAR     : 0x0000000a  EXCCAUSE: 0x00000014  
11:41:48.724 -> EXCVADDR: 0x310813b0  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff  
11:41:48.724 -> 
11:41:48.724 -> Backtrace: 0x310813b1:0x3ffb1f70 0x400d4ce8:0x3ffb1f90 0x400dfa65:0x3ffb1fb0 0x40088b7d:0x3ffb1fd0
11:41:48.724 -> 
11:41:48.724 -> Rebooting...
11:41:48.758 -> ets Jun  8 2016 00:22:57
11:41:48.758 -> 
11:41:48.758 -> rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
11:41:48.758 -> configsip: 0, SPIWP:0xee
11:41:48.758 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
11:41:48.758 -> mode:DIO, clock div:1
11:41:48.758 -> load:0x3fff0018,len:4
11:41:48.758 -> load:0x3fff001c,len:1044
11:41:48.758 -> load:0x40078000,len:8896
11:41:48.758 -> load:0x40080400,len:5816
11:41:48.758 -> entry 0x400806ac

Did I do client object wrong or is it because of missing static keywords? Maybe both? Also I'm not sure if I made correct changes to publish and subscribe objects in setup() since they depend on client object.

I find it to be common that Adafruit libraries crash ESP32's.

Install the ESP Exception Decoder, GitHub - me-no-dev/EspExceptionDecoder: Exception Stack Trace Decoder for ESP8266 and ESP32, then go to the Adafruit site, make a post about the failure, drop in the core debug, backtrace and exception decoder result into the post and let Adafruit solve their issue.

I bought a RPi4, installed MQTT Broker onto the RPi4, and wrote a Python script to distribute MQTT data to several ESP32's in my house and, also, sends the data to my web site's database. Works great!

#include <WiFi.h>
#include <PubSubClient.h>
#include "certs.h"
#include "sdkconfig.h"
#include "esp_system.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/timers.h"
#include "freertos/event_groups.h"
#include <SPI.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BME280.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1351.h>
#include "esp32-hal-psram.h"
#include "esp_himem.h"
#include "time.h"
//
//
hw_timer_t * timer = NULL;
EventGroupHandle_t eg;
#define evtDoDisplay ( 1 << 1 )
#define evtCollectHistory ( 1 << 2 )
#define evtParseMQTT ( 1 << 5 )
#define evtDoTheBME280Thing ( 1 << 8 )
#define evtDoTheHumidityThing ( 1 << 9 )
#define evtDoTheSunLampThing ( 1 << 10 )
#define evtMQTT_Disconnect ( 1 << 11 )
#define evtGroupBits ( evtCollectHistory | evtDoTheBME280Thing | evtDoTheHumidityThing | evtDoTheSunLampThing | evtDoDisplay )
//
Adafruit_SSD1351 tft = Adafruit_SSD1351( 128, 128, GPIO_NUM_27, GPIO_NUM_12, GPIO_NUM_13, GPIO_NUM_14, GPIO_NUM_26 );
////
Adafruit_BME280 bme( GPIO_NUM_5 ); // hardware SPI
////
WiFiClient wifiClient;
PubSubClient MQTTclient( mqtt_server, mqtt_port, wifiClient );
////
////
// memory pointers for variables stored in himem, WROVER PSRAM.
/*
   ePtr = envirtonemtal pointers to float values
   [0] = inside temperature
   [1] = inside humidity
   {2} = inside pressure
   [3] = outside temperature
   [4] = outside humidity
   [5] = outside pressure
   [6] = outside AQ index
*/
float *ePtr;
float *oPressureHistoryPtr;
float *oIAQ_HistoryPtr;
float *oHumidityHistoryPtr;
float *oTemperaturePtr;
////
char *strPayloadPtr;
char *str_eTopicPtr;
////
////
int *ColorPalettePtr;
/*
   int data pointer, used to store globals
   IntDataPtr[0] int DataCells = 50
   IntDataPtr[1] int arrayCellPtr = 0;
   IntDataPtr[2] int humidity set point
   IntDataPtr[3] humidity control enable
   IntDataPtr[4] = sunlamp on
   IntDataPtr[5] = sun lamp enable
*/
int *IntDataPtr;
////
// RTC_DATA_ATTR int bootCount = 0; // assign a memory location in RTC FAST RAM, an experiment remnent
////
////
SemaphoreHandle_t sema_HistoryCompleted;
SemaphoreHandle_t sema_MQTT_Parser;;
SemaphoreHandle_t sema_TheHumidityThing;
SemaphoreHandle_t sema_DoTheSunLampTHing;
SemaphoreHandle_t sema_MQTT_KeepAlive;
////
volatile int iDoTheBME280Thing = 0;
////
void IRAM_ATTR onTimer()
{
  BaseType_t xHigherPriorityTaskWoken;
  iDoTheBME280Thing++;
  if ( iDoTheBME280Thing == 60000 )
  {
    xEventGroupSetBitsFromISR( eg, evtGroupBits, &xHigherPriorityTaskWoken );
    iDoTheBME280Thing = 0;
  }
}
////
void setup()
{
  //
  gpio_config_t io_cfg = {};
  io_cfg.mode = GPIO_MODE_OUTPUT;
  //bit mask of the pins to set
  io_cfg.pin_bit_mask = ( (1ULL << GPIO_NUM_0) | (1ULL << GPIO_NUM_15) | (1ULL << GPIO_NUM_2) );
  //configure GPIO with the given settings
  gpio_config(&io_cfg);
  //pinMode( GPIO_NUM_0, OUTPUT );
  REG_WRITE(GPIO_OUT_W1TS_REG, BIT0);
  //pinMode( GPIO_NUM_2, OUTPUT );
  REG_WRITE(GPIO_OUT_W1TC_REG, BIT12);
  //pinMode( GPIO_NUM_15, OUTPUT );
  REG_WRITE(GPIO_OUT_W1TC_REG, BIT15);
  //
  /* Use 4th timer of 4.
    1 tick 1/(80MHZ/80) = 1us set divider 80 and count up.
    Attach onTimer function to timer
    Set alarm to call timer ISR, every 1000uS and repeat / reset ISR (true) after each alarm
    Start an timer alarm
  */
  timer = timerBegin( 3, 80, true );
  timerAttachInterrupt( timer, &onTimer, true );
  timerAlarmWrite(timer, 1000, true);
  timerAlarmEnable(timer);
  // messageTemp.reserve( 300 );
  // memory allocations for items to be stored in PSRAM
  log_i("before Free PSRAM: %d", ESP.getFreePsram());
  IntDataPtr = (int*)ps_calloc( 15, sizeof(int) );
  IntDataPtr[0] = 50; //const int DataCells = 50;
  oPressureHistoryPtr = (float*)ps_calloc( IntDataPtr[0], sizeof(float) );// oPressureHistoryPtr = (float*)ps_calloc( DataCells, sizeof(float) );
  oIAQ_HistoryPtr = (float*)ps_calloc( IntDataPtr[0], sizeof(float) );//oIAQ_HistoryPtr = (float*)ps_calloc( DataCells, sizeof(float) );
  oHumidityHistoryPtr = (float*)ps_calloc( IntDataPtr[0], sizeof(float) );
  oTemperaturePtr = (float*)ps_calloc( IntDataPtr[0], sizeof(float) );
  ePtr =  (float*)ps_calloc( 15, sizeof(float) );
  strPayloadPtr = (char *)ps_calloc(300, sizeof(char) );
  for ( int i = 0; i < 299; i++ )
  {
    strPayloadPtr[i] = '\0';
    strPayloadPtr[i] = '\0';
  }
  str_eTopicPtr = (char *)ps_calloc(300, sizeof(char) );
  ColorPalettePtr  = (int*)ps_calloc( 10, sizeof(int) );
  log_i("Total heap: %d", ESP.getHeapSize());
  log_i("Free heap: %d", ESP.getFreeHeap());
  log_i("Total PSRAM: %d", ESP.getPsramSize());
  log_i("Free PSRAM: %d", ESP.getFreePsram());
  // popuate color palette for display use
  ColorPalettePtr[0] = 0x0000; //BLACK
  ColorPalettePtr[1] = 0x001F; //BLUE
  ColorPalettePtr[2] = 0xF800; //RED
  ColorPalettePtr[3] = 0x07E0; //GREEN
  ColorPalettePtr[4] = 0x07FF; //CYAN
  ColorPalettePtr[5] = 0xF81F; //MAGENTA
  ColorPalettePtr[6] = 0xFFE0; //YELLOW
  ColorPalettePtr[7] = 0xFFFF; //WHITE
  ColorPalettePtr[8] = 0xFFFFD8; //LIGHTYELLOW
  ColorPalettePtr[9] = 0xFF8040; //BROWN
  //
  ePtr[2] = 50; // set a default humidity level
  ePtr[3] = 0; // set humidity control to off
  //
  eg = xEventGroupCreate();
  SPI.begin();
  bme.begin();
  while ( !bme.begin() )
  {
    log_i( "Waiting on response from BME280");
    vTaskDelay( 1000 );
  }
  tft.begin();
  tft.fillScreen(0x0000);
  //
  sema_MQTT_Parser = xSemaphoreCreateBinary();
  sema_HistoryCompleted = xSemaphoreCreateBinary();
  sema_TheHumidityThing = xSemaphoreCreateBinary();
  sema_DoTheSunLampTHing = xSemaphoreCreateBinary();
  sema_MQTT_KeepAlive = xSemaphoreCreateBinary();
  xSemaphoreGive( sema_DoTheSunLampTHing );
  xSemaphoreGive( sema_HistoryCompleted );
  xSemaphoreGive( sema_TheHumidityThing );
  xSemaphoreGive( sema_MQTT_KeepAlive ); // found keep alive can mess with a publish, stop keep alive during publish

part 2

 ////
  xTaskCreatePinnedToCore( fparseMQTT, "fparseMQTT", 7000, NULL, 5, NULL, 1 ); // assign all to core 1, WiFi in use.
  xTaskCreatePinnedToCore( fCollectHistory, "fCollectHistory", 10000, NULL, 4, NULL, 1 );
  xTaskCreatePinnedToCore( MQTTkeepalive, "MQTTkeepalive", 7000, NULL, 2, NULL, 1 );
  xTaskCreatePinnedToCore( fUpdateDisplay, "fUpdateDisplay", 50000, NULL, 5, NULL, 1 );
  xTaskCreatePinnedToCore( DoTheBME280Thing, "DoTheBME280Thing", 20000, NULL, 3, NULL, 1 );
  xTaskCreatePinnedToCore( fDoTheHumidityThing, "fDoTheHumidityThing", 2000, NULL, 3, NULL, 1 );
  xTaskCreatePinnedToCore( fDoTheSunLampThing, "fDoTheSunLampThing", 2000, NULL, 3, NULL, 1 );
  xTaskCreatePinnedToCore( fMQTT_Disconnect, "fMQTT_Disconnect", 2000, NULL, 2, NULL, 1 );
  ////
} //setup()
////
void fMQTT_Disconnect( void * parameter )
{

  for (;;)
  {
    xEventGroupWaitBits (eg, evtMQTT_Disconnect, pdTRUE, pdTRUE, portMAX_DELAY );
    xSemaphoreTake( sema_MQTT_KeepAlive, portMAX_DELAY ); //
    MQTTclient.disconnect();
    xSemaphoreGive( sema_MQTT_KeepAlive );
  }
  vTaskDelete( NULL );
}
////
void GetTheTime()
{
  char* ntpServer = "2.us.pool.ntp.org";
  int gmtOffset_sec = -(3600 * 7 );
  int daylightOffset_sec = 3600;
  configTime(gmtOffset_sec, daylightOffset_sec, ntpServer);
  printLocalTime();
}
////
// http://www.cplusplus.com/reference/ctime/strftime/
////
int getHour()
{
  struct tm timeinfo;
  getLocalTime(&timeinfo);
  char _time[ 5 ];
  strftime( _time, 80, "%T", &timeinfo );
  return String(_time).toInt();
}
////
void printLocalTime() {
  struct tm timeinfo;
  getLocalTime(&timeinfo);
  char _time[ 80 ];
  strftime( _time, 80, "%T", &timeinfo );
  log_i( "%s", _time);
  //  }
}
////
void fDoTheSunLampThing( void * parameter )
{
  //int x = gpio_get_level( GPIO_NUM_15 ); // reads gpio pin state returns an int
  // IntDataPtr[4] = sunlamp on manual mode, automatic mode off for manual mode to work
  // IntDataPtr[5] = sun lamp enable automatic mode
  bool AlreadyOn = false;
  for (;;)
  {
    xEventGroupWaitBits (eg, evtDoTheSunLampThing, pdTRUE, pdTRUE, portMAX_DELAY );
    vTaskDelay( 6 );
    xSemaphoreTake( sema_DoTheSunLampTHing, portMAX_DELAY );
    // sun lamp on/off auto disabled
    if ( (IntDataPtr[4] == 1) && (IntDataPtr[5] == 0)  )
    {
      if ( !AlreadyOn )
      {
        REG_WRITE(GPIO_OUT_W1TS_REG, BIT15);
        AlreadyOn = true;
      } else {
        REG_WRITE(GPIO_OUT_W1TC_REG, BIT15);
        AlreadyOn = false;
      }
    } else if ( (IntDataPtr[4] == 0) && (IntDataPtr[5] == 1) ) // light off auto enabled
    {
      int _hour = getHour();
      if ( (_hour >= 7) && (_hour <= 17) )
      {
        //log_i( "hour %d", _hour );
        REG_WRITE(GPIO_OUT_W1TS_REG, BIT15);
        AlreadyOn = true;
      }
      if ( (_hour < 7) || (_hour > 16) )
      {
        REG_WRITE(GPIO_OUT_W1TC_REG, BIT15);
        AlreadyOn = false;
      }
    } else {
      REG_WRITE(GPIO_OUT_W1TC_REG, BIT15);
      AlreadyOn = false;
    }
    xSemaphoreGive( sema_DoTheSunLampTHing );
    //log_i( "fDoTheSunLampThing high watermark %d",  uxTaskGetStackHighWaterMark( NULL ) );
  }
  vTaskDelete( NULL );
} //void fCollectHistory( void * parameter )
////
// send a signal out to relay to energize or de-energize humidifier based on set point
void fDoTheHumidityThing( void * parameter )
{
  for (;;)
  {
    xEventGroupWaitBits (eg, evtDoTheHumidityThing, pdTRUE, pdTRUE, portMAX_DELAY );
    vTaskDelay( 4 );
    xSemaphoreTake( sema_TheHumidityThing, portMAX_DELAY );
    if ( IntDataPtr[3] == 1 )
    {
      if ( IntDataPtr[2] + 1 < (int)ePtr[2] )
      {
        REG_WRITE(GPIO_OUT_W1TC_REG, BIT2);
      }
      if ( (IntDataPtr[2] - 1) > (int)ePtr[2] )
      {
        // energize humidifier
        REG_WRITE(GPIO_OUT_W1TS_REG, BIT2);
      }
    } else {
      //de energize humidifier
      REG_WRITE(GPIO_OUT_W1TC_REG, BIT2);
    }
    xSemaphoreGive( sema_TheHumidityThing );
    //log_i( "fDoTheHumidityThing high watermark %d",  uxTaskGetStackHighWaterMark( NULL ) );
  }
  vTaskDelete( NULL );
} //void fCollectHistory( void * parameter )
////
/*
   Collect history information
   task triggered by hardware timer once a minute
   stores history into PSRAM
   Using a semaphore to protect the PSRAM from multiple tasks access the same PSRM locations
*/
void fCollectHistory( void * parameter )
{
  int StorageTriggerCount = 59;
  for (;;)
  {
    xEventGroupWaitBits (eg, evtCollectHistory, pdTRUE, pdTRUE, portMAX_DELAY );
    StorageTriggerCount++; //triggered by the timer isr once a minute count 60 minutes
    if ( StorageTriggerCount == 60 )
    {
      xSemaphoreTake( sema_HistoryCompleted, portMAX_DELAY );
      oPressureHistoryPtr[IntDataPtr[1]] = ePtr[5];
      oIAQ_HistoryPtr[IntDataPtr[1]] = ePtr[6];
      oHumidityHistoryPtr[IntDataPtr[1]] = ePtr[4];
      oTemperaturePtr[IntDataPtr[1]] = ePtr[3];
      IntDataPtr[1]++;
      xSemaphoreGive( sema_HistoryCompleted );
      //log_i( "pointer %d stored %f", IntDataPtr[1] - 1, oPressureHistoryPtr[IntDataPtr[1] - 1] );
      if ( IntDataPtr[1] == IntDataPtr[0] )
      {
        IntDataPtr[1] = 0;
      }
      StorageTriggerCount = 0;
    }
    //log_i( ">>>>>>>>>>>>ARRAY CELL POINTER %d storagetriggercount %d", IntDataPtr[1] - 1, StorageTriggerCount );
    // log_i( " high watermark %d",  uxTaskGetStackHighWaterMark( NULL ) );
  }
  vTaskDelete( NULL );
} //void fCollectHistory( void * parameter )
////
// Using a semaphore to protect the PSRAM from multiple tasks access the same PSRM locations
////
void fUpdateDisplay( void * parameter )
{
  // Color definitions
  // http://www.barth-dev.de/online/rgb565-color-picker/
  /*
    ColorPalettePtr[0] = 0x0000; //BLACK
    ColorPalettePtr[1] = 0x001F; //BLUE
    ColorPalettePtr[2] = 0xF800; //RED
    ColorPalettePtr[3] = 0x07E0; //GREEN
    ColorPalettePtr[4] = 0x07FF; //CYAN
    ColorPalettePtr[5] = 0xF81F; //MAGENTA
    ColorPalettePtr[6] = 0xFFE0; //YELLOW
    ColorPalettePtr[7] = 0xFFFF; //WHITE
    ColorPalettePtr[8] = 0xFFFFD8; //LIGHTYELLOW
    ColorPalettePtr[9] = 0xFF8040; //BROWN
  */

part 3

for (;;)
  {
    xEventGroupWaitBits (eg, evtDoDisplay, pdTRUE, pdTRUE, portMAX_DELAY ); //
    vTaskDelay( 10 );
    tft.fillScreen( ColorPalettePtr[0] );
    tft.setTextColor( ColorPalettePtr[7] );
    tft.setCursor( 0, 0 );
    tft.print( "Inside"  );
    tft.setTextColor( ColorPalettePtr[3] );
    tft.setCursor( 0, 15 );
    tft.print( "Temp: " + String(ePtr[0]) + "C " + String((ePtr[0] * 9 / 5) + 32) + "F"  );
    tft.setCursor( 0, 25 );
    tft.print( "Humidity " + String(ePtr[2]) + "%" );
    //
    tft.setTextColor( ColorPalettePtr[7] );
    tft.setCursor( 0,  40 );
    tft.print( "Outside" );
    tft.setTextColor( ColorPalettePtr[6] );
    tft.setCursor( 0, 55 );
    tft.print( "Temperature: " + String(ePtr[3]) + "F" );
    tft.setTextColor( ColorPalettePtr[2] );
    tft.setCursor( 0, 65 );
    tft.print( "Humidity " + String(ePtr[4]) + "%" );
    tft.setCursor( 0, 75 );
    tft.setTextColor( ColorPalettePtr[1] );
    tft.print( "Pres. " + String(ePtr[5]) + "mmHg" );
    tft.setCursor( 0, 86 );
    //set the color of the value to be displayed
    if ( ePtr[6] < 51.0f )
    {
      tft.setTextColor( ColorPalettePtr[3] );
    }
    if ( (ePtr[6] >= 50.0f) && (ePtr[6] <= 100.0f) )
    {
      tft.setTextColor( ColorPalettePtr[6] );
    }
    if ( (ePtr[6] >= 100.0f) && (ePtr[6] <= 150.0f) )
    {
      tft.setTextColor( ColorPalettePtr[9] );
    }
    if ( (ePtr[6] >= 150.0f) && (ePtr[6] <= 200.0f) )
    {
      tft.setTextColor( ColorPalettePtr[2] );
    }
    if ( (ePtr[6] >= 200.00f) && (ePtr[6] <= 300.0f) )
    {
      tft.setTextColor( ColorPalettePtr[5] );
    }
    if ( (ePtr[6] > 300.0f) )
    {
      tft.setTextColor( ColorPalettePtr[7] );
    }
    tft.print( "AQ Index " + String(ePtr[6]) );
    tft.setTextColor( ColorPalettePtr[1] ); //set graph line color
    int rowRef = 110;
    int hRef = int( oPressureHistoryPtr[0] );
    int nextPoint = 2;
    int nextCol = 0;
    xSemaphoreTake( sema_HistoryCompleted, portMAX_DELAY );
    for (int i = 0; i < IntDataPtr[0]; i++)
    {
      int hDisplacement = hRef - int( oPressureHistoryPtr[i] ); // cell height displacement from base line
      tft.setCursor( nextCol , (rowRef + hDisplacement) );
      tft.print( "_" );
      nextCol += nextPoint;
    }
    tft.setCursor( (IntDataPtr[1] * nextPoint), (rowRef + 3) );
    tft.print( "I" );
    xSemaphoreGive( sema_HistoryCompleted );
    //     log_i( "fUpdateDisplay MEMORY WATERMARK %d", uxTaskGetStackHighWaterMark(NULL) );
    // xEventGroupSetBits( eg, evtConnect ); // trigger task
  }
  vTaskDelete( NULL );
} // void fUpdateDisplay( void * parameter )
////
////
void DoTheBME280Thing( void *pvParameters )
{
  for (;;)
  {
    xEventGroupWaitBits (eg, evtDoTheBME280Thing, pdTRUE, pdTRUE, portMAX_DELAY ); //
    // log_i( "Signal strength %d", WiFi.RSSI() );
    vTaskDelay( 2 );
    if ( !isnan(bme.readTemperature()) )
    {
      ePtr[0] = bme.readTemperature();
      ePtr[1] = bme.readPressure() / 133.3223684f; // mmHg
      ePtr[2] = bme.readHumidity();
      if ( MQTTclient.connected() ) // broker online, then send stuff
      {
        xSemaphoreTake( sema_MQTT_KeepAlive, portMAX_DELAY ); // do not send whiles keep alive is in progress and pause keep-alive during send
        MQTTclient.publish( topicInsideTemp, String(ePtr[0]).c_str() );
        vTaskDelay( 2 ); // gives the Raspberry Pi 4 time to receive the message and process
        MQTTclient.publish( topicInsideHumidity, String(ePtr[1]).c_str() );
        vTaskDelay( 2 ); // no delay and RPi is still processing previous message
        MQTTclient.publish( topicInsidePressure, String(ePtr[2]).c_str() );
        xSemaphoreGive( sema_MQTT_KeepAlive );
      }
    }
    //log_i( "DoTheBME280Thing high watermark %d",  uxTaskGetStackHighWaterMark( NULL ) );
  }
  vTaskDelete ( NULL );
}
////