A few questions about MQTT beyond initial publish/subscribe

Leveraging the PubSubClient.h - Thanks to knolleary

I’m able to get temperature readings from my device to a cloud service.
I’m also able to get subscription work (a hello from PC makes it onto the IoT Device)

My questions are, if someone could help.

  1. Does client.loop() call the callback function?
    If I don’t have the client.loop() then my sketch doesn’t work, I expect it does, but I would have thought with previous subscribe and connect messages it would work without.
    client.subscribe(TopicSwitch);
    client.loop();// This will check the callback function to see if there is a message - Breaks if not here……

If I issue the following command, will it stay subscribed to this topic as long as my MQTT connection is alive
client.subscribe(TopicSwitch);

It again, seems like I must use this subscribe just before the client.loop() to get any data returned.

If I wanted to subscribe and print messages from two topics, would I have to subscribe AND run the client.loop for this?
client.subscribe(TopicSwitch1);
client.loop()
client.subscribe(TopicSwitch2);
client.loop()

I’ve seen people use
client.subscribe(“2177800/1206/arduinoin”);

In their Setup() but I would like to subscribe to a few topics

and then client.loop() with separate topics

for example

client.loop() — Check Topic 1 for Alert
client.loop() - Check Topic 2 for Alert

  • Note, this does seem to work if I subscribe to a topic just before running client.loop() but wonder if there is a better way

How can I get the value of the cstring out of the callback function into my loop()

I would like to use the value to turn off/on a switch?
Tried to “return cstring” but that didn’t work.

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  Serial.println();
  Serial.println("Attempting Print entire Payload");
  char *cstring = (char *) payload;
  cstring[length] = '\0';    // Adds a terminate terminate to end of string based on length of current payload
  Serial.println(cstring);    
  Serial.println();
//  return cstring;
}
Serial.println("Testing TopicSwitch");
  client.subscribe(TopicSwitch); 
  client.loop();// This will check the callback function to see if there is a message"
  1. Does client.loop() call the callback function?
    If I don't have the client.loop() then my sketch doesn't work, I expect it does, but I would have thought with previous subscribe and connect messages it would work without.
    client.subscribe(TopicSwitch);
    client.loop();// This will check the callback function to see if there is a message - Breaks if not here…..

No. client.loop() does not check the callback function. It checks to see if an event has happened. If one has happened, and there is a callback registered for that event, it calls the callback.

If I issue the following command, will it stay subscribed to this topic as long as my MQTT connection is alive
client.subscribe(TopicSwitch);

I would expect it to.

It again, seems like I must use this subscribe just before the client.loop() to get any data returned.

That seems odd, but you haven't provided any code that illustrates this, or any proof.

If I wanted to subscribe and print messages from two topics, would I have to subscribe AND run the client.loop for this?
client.subscribe(TopicSwitch1);
client.loop()
client.subscribe(TopicSwitch2);
client.loop()

No. You'd have two subscribe() method calls (I'd expect to see them in setup()) and one loop() method call (in loop()).

How can I get the value of the cstring out of the callback function into my loop()

Think of a callback like this. You are home cooking dinner. The doorbell rings. You do something about that (whatever the callback function says to do). How you get information from the person at the door to the meal you are cooking is up to you. Copying the data to a global variable/array is one way.

I would like to use the value to turn off/on a switch?

Would you? Why not do that in the callback function?

Tried to "return cstring" but that didn't work.

From a function whose contract says it returns nothing? I'm not surprised it "didn't work".

PaulS:
No. client.loop() does not check the callback function. It checks to see if an event has happened. If one has happened, and there is a callback registered for that event, it calls the callback.
I would expect it to.

Well, I know it's calling the callback because without it, I get nothing :slight_smile: How would I know for sure if a callback was registered. I've never done anything like this in the past, so it's new to me.

PaulS:
That seems odd, but you haven't provided any code that illustrates this, or any proof.

Sorry, I wasn't clear. I must have the subscribe before the client.loop() because I haven't subscribed anywhere else in the code and the reason is that I don't know how to subscribe to different topics at the same time and use a single client.loop() to check for a response. Maybe that's something I need to give a go

PaulS:
No. You'd have two subscribe() method calls (I'd expect to see them in setup()) and one loop() method call (in loop()).

Ok. Will give this a try. I think it's more work to have to differentiate which callback for which topic rather than just subscribing, running the client.loop() then subscribing again to the next topic. Probably because I dont' know how to return the values out of the callback.

PaulS:
Think of a callback like this. You are home cooking dinner. The doorbell rings. You do something about that (whatever the callback function says to do). How you get information from the person at the door to the meal you are cooking is up to you. Copying the data to a global variable/array is one way.

This makes sense. I'll need to figure how I can yell to the front door from the kitchen :slight_smile:

PaulS:
Would you? Why not do that in the callback function?

Because I thought the callback function is just temporary and I didn't know how to get the data out of the callback.

PaulS:
From a function whose contract says it returns nothing? I'm not surprised it "didn't work".

Yes, that's why I was trying the "return cstring". I know how to create a global variable. Can I just create a global variable and write to it from within callback and use it in loop() ?

That would seem too easy?

I trying to declare callbackstring as global, use it in the callback function and then display it in loop() but that didn't work. Displaying the value of callbackString inside the callback worked.

char callbackString;

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.println("Attempting Print entire Payload");
  char *callbackString = (char *) payload;
  callbackString[length] = '\0';    // Adds a terminator to end of string based on length of current payload
  // null terminate payload
  // char * payload = (char *)message.payload;
  // payload[message.payloadlen] = '\0';
  // messageReceived(String(topic), String(payload), (char*)message.payload, (unsigned int)message.payloadlen);
  //  end Null Terminate
  Serial.println(callbackString);    
  Serial.println();
//  return callbackString;
}

loop()
 Serial.println("Displaying callbackString from inside loop yields");
  Serial.print(callbackString);

How would I know for sure if a callback was registered.

Well, no one else is registering callbacks in your code for you...

I must have the subscribe before the client.loop() because I haven't subscribed anywhere else in the code and the reason is that I don't know how to subscribe to different topics at the same time and use a single client.loop() to check for a response. Maybe that's something I need to give a go

I would expect to be able to register to be called for several events, and then, in the callback determine why I was called.

Probably because I dont' know how to return the values out of the callback.

You can't. That is not what a callback is for. The callback is where you are supposed to deal with the event. For instance, when you press the Post button, a callback happens. You would expect that everything that was supposed to happen would happen in that function. You would not, for instance, expect that the data in the text field was copied to the clipboard and you needed to paste that somewhere else.

I'll need to figure how I can yell to the front door from the kitchen

While you are at the front door?

Because I thought the callback function is just temporary

No, it's permanent code.

and I didn't know how to get the data out of the callback.

Once again, I don't think you should be thinking this way. The callback is supposed to completely handle the event.

Can I just create a global variable and write to it from within callback and use it in loop() ?

Of course.

I trying to declare callbackstring as global, use it in the callback function and then display it in loop() but that didn't work.

For several reasons. One is that the global callbackstring is a char, while the local one is a pointer. Second is that there IS a local one, where you seem to have meant to do was value the global variable. Third, and most important is that you are trying to point to space on the stack that is going to be clobbered when the function returns.

char callbackCrap[100];

void callback(char* topic, byte* payload, unsigned int length)
{
   memcpy(callbackCrap, (char *)payload, length);
   callbackCrap[length] = '\0';

   // do whatever else needs doing
}

void loop()
{
   // Here, you could, if you just absolutely must, use the text from the callback function

   callbackCrap[0] = '\0'; // Then wipe it out
}

Thanks PaulS. I've learned some new concepts this evening.

PaulS:
I would expect to be able to register to be called for several events, and then, in the callback determine why I was called.
The callback is where you are supposed to deal with the event. For instance, when you press the Post button, a callback happens. You would expect that everything that was supposed to happen would happen in that function. You would not, for instance, expect that the data in the text field was copied to the clipboard and you needed to paste that somewhere else.

That was my misunderstanding. I thought the callback was just to capture the data. I didn't realize my callback section could wind up being quite large if I subscribed to multiple topics. Because I will have to create if conditions to match on topic, considering there is only one callback for the client.loop()

PaulS:
For several reasons. One is that the global callbackstring is a char, while the local one is a pointer. Second is that there IS a local one, where you seem to have meant to do was value the global variable. Third, and most important is that you are trying to point to space on the stack that is going to be clobbered when the function returns.

Yes, indeed what I wanted to do is get the bytes from the payload into the string called Callbackstring and be able to use it later to do something.
For the 3rd, and most clobbering, I was thinking that if I wrote to the global variable from inside the callback that it would persist. I knew local variables were deleted and memory freed but I thought I could get data out this way.

Can you explain how to value the global variable
"where you seem to have meant to do was value the global variable"
Is it in your code below?

PaulS:

char callbackCrap[100];

void callback(char* topic, byte* payload, unsigned int length)
{
  memcpy(callbackCrap, (char *)payload, length);
  callbackCrap[length] = '\0';

// do whatever else needs doing
}

void loop()
{
  // Here, you could, if you just absolutely must, use the text from the callback function

callbackCrap[0] = '\0'; // Then wipe it out
}

Thanks again for this. I'll attempt to do everything I need in the callback and just save this as a option if I must.

Because I will have to create if conditions to match on topic, considering there is only one callback for the client.loop()

What you should do is just have the callback function look at the topic, and decide what function to call to deal with the payload and length associated with that topic.

I knew local variables were deleted and memory freed but I thought I could get data out this way.

Not by using a pointer to memory that is going to get clobbered.

Thanks again Paul.

Just a couple of things in case others stumble upon this thread

You can indeed subscribe to multiple topics, and do it in setup() however, make sure you are connected to the MQTT broker before you subscribe like

if (client.connect("uniqureIDhere", mqtt_user, mqtt_password)) {
      Serial.println("connected");
    } 
   client.subscribe(TopicSwitch); 
   client.subscribe(TopicTemp);

Do note that if you can’t connect before your loop() starts, you cannot subscribe so best make sure you are subscribed.

Or better, put it in your reconnect() function so that if you connect to MQTT, you subscribe.

However, a few moments later and I realized I cannot seem to match the *char that is topic with

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  Serial.println();
  char *callbackString = (char *) payload;
  callbackString[length] = '\0';    // Adds a terminator to end of string based on length of current payload
  if (topic == "/home/ESP8266-Sensor1/Switch") {
    Serial.println("Matched /Switch....");
                             } //end If
}

However, a few moments later and I realized I cannot seem to match the *char that is topic with

Of course not. strcmp() is how you compare strings.

PaulS:
Of course not. strcmp() is how you compare strings.

The more I get working thanks to your help, the more I realize I don't know :slight_smile:

This topic comparison for multiple subscribed switch topics here

int strcomparison = strcmp(topic, TopicSwitch);
  if (strcomparison == 0) {
    Serial.println("Matched Switch1 Topic");
                             } //end If
strcomparison = strcmp(topic, TopicSwitch2);
  if (strcomparison == 0) {
    Serial.println("Matched Switch2 Topic");
                             } //end If

Here is the working code for everyone. Maybe it’ll help someone out in the future.
Thanks PaulS and others for the help.

This is a temperature sensor connected to Pin D2, MQTT publish function to get the temperature to cloudmqtt, and a subscribe function for a switch (LED, servo, whatever you need)

// ESP8266 DS18B20 ArduinoIDE Thingspeak IoT Example code
// http://vaasa.hacklab.fi
//
// https://github.com/milesburton/Arduino-Temperature-Control-Library
// https://gist.github.com/jeje/57091acf138a92c4176a

// copied from here - https://vaasa.hacklab.fi/2016/02/06/esp8266-on-nodemcu-board-ds18b20-arduinoide-thingspeak/

#include <PubSubClient.h>
#include <OneWire.h>
#include <ESP8266WiFi.h>
// #include <ESP8266WebServer.h>
#include <DallasTemperature.h>

/////////////////////
// Pin Definitions //
/////////////////////
#define ONE_WIRE_BUS D2

OneWire oneWire(ONE_WIRE_BUS);
DallasTemperature DS18B20(&oneWire);

//////////////////////
// WiFi Definitions //
//////////////////////
const char* ssid     = “yours”;
const char* pass = “yours”;
char callbackString;

/////////////////////
//     Variables   //
//////////////////////
char temperatureString[6];
char LasttemperatureString[6];

/////////////////////
// MQTT Definitions //
//////////////////////
#define mqtt_server “____.cloudmqtt.com"
#define mqtt_user “_____”
#define mqtt_password "_____"
#define TopicTemp "/home/ESP8266-Sensor1/Temp"
#define TopicSwitch "/home/ESP8266-Sensor1/Switch"

///////////////////////
// Other Definitions //
///////////////////////
WiFiClient espClient;
PubSubClient client(espClient);


void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  Serial.print("Length is ");
  Serial.print(length);
  Serial.println();
  char *callbackString = (char *) payload;
  callbackString[length] = '\0';    // Adds a terminator to end of string based on length of current payload
  int strcomparison = strcmp(topic, TopicSwitch);
  if (strcomparison == 0) {
    Serial.println("Matched Switch");
    if (length == 2) {
      digitalWrite(D1, HIGH);
    }
    if (length == 3) {
      digitalWrite(D1, LOW);
    }
                         } //end If
  strcomparison = strcmp(topic, TopicTemp);
  if (strcomparison == 0) {
    Serial.println("Matched Temp");
                             } //end If
}

void setup(void){
  Serial.begin(115200);
  Serial.println("");
  pinMode(D1, OUTPUT);
  WiFi.begin(ssid, pass);
  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(100);
    Serial.print(".");
  }
  
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

  DS18B20.begin();

   //MQTT Stuff here
   client.setServer(mqtt_server, 15557);  // 15557 is the port given my cloudmqtt
   client.setCallback(callback);
}

float getTemperature() {
  Serial.println("Entered getTemperature()");
  float temp;
  DS18B20.requestTemperatures();
  temp = DS18B20.getTempCByIndex(0);
  delay(100);
  return temp;
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
       if (client.connect("tisnow_1313136575", mqtt_user, mqtt_password)) {
      Serial.println("connected");
      client.subscribe(TopicSwitch); 
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void loop() {
 {
  // First make sure MQTT is connected 
  if (!client.connected()) {
    reconnect();
  }

// Starting to read and print Temperature Readings
  Serial.println("Entering loop()");
  float temperature = getTemperature();  // returned data from getTemperature added to temperature
  dtostrf(temperature, 2, 2, temperatureString);
  // send temperature to the serial console
  Serial.println("Attempting to print temperatureString in loop()");
  Serial.println(temperatureString);

 
// Start MQTT Publish //
 
  Serial.println("Attempting a client.publish to " TopicTemp);
  // client.connect("ESP8266Client", mqtt_user, mqtt_password);
  client.publish(TopicTemp, temperatureString);
  } // Close If


// Start MQTT Subscription Functions  //
//  Serial.println("Attempting to subscribe to " TopicSwitch);
  Serial.println("Retrieving any new messages in subscribed topics");  // Subscribe works, but variable does not
  client.loop(); // This will check the callback function to see if there is a message"


  Serial.println("------------------------------------");
  delay(3000);

}  // End of Loop

Thanks for sharing the code with us, I was wondering how adding 4 switches which would involve appending 3 more separate topics plus 2 sensor topics (humidity and temperature) would be all implemented, any idea how how to have all the 6 topics in on single sketch?

zkopf:
Thanks for sharing the code with us, I was wondering how adding 4 switches which would involve appending 3 more separate topics plus 2 sensor topics (humidity and temperature) would be all implemented, any idea how how to have all the 6 topics in on single sketch?

You are welcome. Many have helped me in my learning process. I still haven't figured out how to make a 18650 battery last for a few days without really long sleep cycles which defaults the purpose of a sensor to wait seconds at a time (for door open/close)

But anyways.

In order to add 4 switches, I think you could use either 4 distinct topics, or a single topic and publish different values and let your MQTT cloud filters filter them how you wish to take option.
EG Switches is the topic
S1On or S1Off
S2On or S2Off

But id' probably just create multiple topics.