Include ThingsBoard PubSubClient

Projects Networking, Protocols, and Devices
1

PubSubClient hasn't been updated since 2020 and doesn't work.

However, ThingBoard have a fork that is being maintained: GitHub - thingsboard/pubsubclient: Fork of a client library for the Arduino that provides support for MQTT.

The Library Manager can install the ThingsBoard version which appears as TPPubSubClient and gets put at

libraries\TBPubSubClient\src

but

#include <PubSubClient.h>

uses the broken old version.

Both files are called PubSubClient.h.

How do you get a sketch to use the ThingsBoard version?

2

That is a very sweeping statement

What exactly "doesn't work"

Which version of PubSubClient are you using ?

3

Version 2.8 from 2020.
It can publish, but subscribe never receives anything.

4

The last time I tried it worked OK for me

Delete the version that you don't want from the sketchbook libraries folder or use the Library Manager to remove it

I would be interested in hearing the results of trying the TBPubSubClient version

5

Shh don't tell my computer, it is and has been using PubSubClient for a few years along with a few hundred other folks.

6

The Library manager doesn't have an option to remove PubSubClient.

Deleted the folder.

Still not receiving messages, so the ThingsBroard version is broken too.
I guess you get what you pay for.

Gut feeling is that it's something to do with timing and loop because if you spam the topic really fast then sometimes a message gets through.

A workaround is to run this on debian to read the topic and expose it to the network.

while true; do nc -l -p 1884 -q 1 -c "mosquitto_sub -h localhost -t \"foo/bar/buz\" -C 1 -F \"%p\"" ; done

and then read it in the Arduino like this

     // Try to get the new temp.
      wifi_client.connect("192.168.0.113", 1884);
      delay(500);
      bool waited = 0;
      for(int i=0; i < 12 && (!wifi_client.connected() || !wifi_client.available()); i++){
        Serial.print(i);
        Serial.print("... ");
        delay(500);
        waited = true;
      }
      if( waited ){ Serial.println(); }
      StaticJsonDocument<32> jDoc;
      if (wifi_client.available()) {
        deserializeJson(jDoc, wifi_client);
        foo = jDoc["foo"];
        Serial.print(current_foo);
        Serial.print(" -> ");
        Serial.println(foo);
        if(foo> 0 & current_foo == 0 ){ current_foo = foo; }
      }
      else {
        Serial.println("No data from 1884");
      }
      wifi_client.stop();

A problem with this is that you can only see the latest retained message on the topic in MQTT therefore the value must represent the desired state rather than an instruction to change state; comparing foo to the current_foo determines whether an action must be taken.

7

I am willing to bet that it isn't

Please post your full sketch that does not receiev data from MQTT

1 Like
8

The capability was added in Arduino IDE 2.x, but if you are still using Arduino IDE 1.x then indeed the Library Manager didn't provide an uninstall capability at that time and the only way to uninstall libraries is by manually deleting their installation folder as you did.

9

Which version of the IDE are you using ?

10

Are you calling PubSubClient::loop

in your loop, as shown in the examples?

11 
#include <WiFi.h>
#include <PubSubClient.h>

const char *ssid = "196883";
const char *password = "TOP_SECRET_PASSWORD_DONT_TELL_PUTIN";

const char * mqtt_broker = "192.168.0.113";
const int mqtt_port = 1883;
//const char mqtt_topic[] = "zigbee2mqt/#";
//const char mqtt_topic[] = "zigbee2mqtt/Home/+/Thermostat";
const char mqtt_topic[] = "zigbee2mqtt/Home/Hall/Thermostat";

WiFiClient wifi_client;

PubSubClient mqtt_client(wifi_client);


void setup() {
  Serial.begin(115200);
  Serial.printf("Connecting to %s ", ssid);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println(" CONNECTED");

  mqtt_client.setServer(mqtt_broker, mqtt_port);
  mqtt_client.setCallback(mqtt_callback);
  while (!mqtt_client.connected()) {
    Serial.println("-- connect_mqtt --");
    if (mqtt_client.connect("arduinoClient")) {
      return;
    } else {
      Serial.print("  failed, rc=");
      Serial.print(mqtt_client.state());
      Serial.println(". Retrying...");
      delay(2000);
    }
  }
}

void mqtt_callback(char* topic, byte* payload, unsigned int length) {
  Serial.println();
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");

  Serial.println("Payload:");
  for (int i=0;i<length;i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
}

int n = 0;
int m = 0;
void loop() {
  if(n > 1000000) {
    Serial.println(m);
    m++;
    n = 0;
  }
  n++;
  mqtt_client.loop();
}

Nothing is ever received.

12

Typo here: missing second "t". This wildcard topic might have worked.

Unless you configured it differently, neither of these follow the topic format, as documented.

13

Surely

const char mqtt_topic[] = "zigbee2mqtt/Home/Hall/Thermostat";

should work?
It works with mosquitto_sub.

Using zigbee2mqtt/Home/# still gets nothing.

14

Still don't want to try "zigbee2mqtt/#"?

One other thing, which I missed last time: there's no mqtt_client.subscribe call.

Just as a baseline, here is a tweaked version of their "mqtt_basic" example

#include <WiFi.h>
#include <PubSubClient.h>
#include "arduino_secrets.h"

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i=0;i<length;i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
}

WiFiClient wifi;
PubSubClient pubsub(wifi);

void reconnect() {
  // Loop until we're reconnected
  while (!pubsub.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (pubsub.connect("arduinoClient-867-5309")) {
      // Once connected, publish an announcement...
      // pubsub.publish("outTopic","hello world");
      // ... and resubscribe
      pubsub.subscribe("one/two/three/#");
      Serial.println("connected and subscribed");
    } else {
      Serial.print("failed, rc=");
      Serial.print(pubsub.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup()
{
  Serial.begin(115200);

  pubsub.setServer("test.mosquitto.org", 1883);
  pubsub.setCallback(callback);

  WiFi.begin(SECRET_SSID, SECRET_PASS);
  for (int i = 0; WiFi.status() != WL_CONNECTED; i++) {
    Serial.print(i % 10 ? "." : ".\n");
    delay(100);
  }
}

void loop()
{
  if (!pubsub.connected()) {
    reconnect();
  }
  pubsub.loop();
}

at test.mosquitto.org. Then to test, which worked for me

$ mosquitto_pub -h test.mosquitto.org -p 1883 -t 'one/two/three/four' -m "hola: $(date)"
15

Duh, yes, subscribe was missing.

Subscribing to zigbee2mqtt/# receives some but not all topics, for example:

zigbee2mqtt/Home/Hall/PIR len 137
zigbee2mqtt/Home/LivingRoom/Socket/Table/set len 15
zigbee2mqtt/Home/Hall/Socket/Lantern/set len 15
zigbee2mqtt/lux len 63
zigbee2mqtt/Home/Kitchen/Counter/set len 15
zigbee2mqtt/Home/Kitchen/Counter len 56
zigbee2mqtt/Home/Kitchen/Counter len 56
zigbee2mqtt/Home/Kitchen/Counter/set len 15
zigbee2mqtt/Home/Kitchen/Counter len 56
zigbee2mqtt/Home/Kitchen/Counter len 56
zigbee2mqtt/Home/scene len 196
zigbee2mqtt/Home/Kitchen/Counter len 56
zigbee2mqtt/Home/Hall/PIR len 138
zigbee2mqtt/lux len 63

But others -- such as zigbee2mqtt/Home/+/Thermostat are never seen, even though they appear in mosquitto_sub.

16

I can get the messages to appear on the ESP32 if I run this on linux to repeat them to a different topic.

while true  # Keep an infinite loop to reconnect when connection lost/broker unavailable
do
    echo "Connecting"
    mosquitto_sub -h localhost -t "zigbee2mqtt/Home/+/Thermostat" -F "%t %p" | while read -r payload
    do
        topic=$(echo "$payload" | cut -d ' ' -f 1)
        room=$( echo "$topic" | awk -F '/' '{print $3}' )
        msg=$(echo "$payload" | cut -d ' ' -f 2-)
        echo "TOPIC $topic"
        mosquitto_pub -h localhost -p 1883 -t "thermostat/$room" -m "XXX"
    done
    sleep 1  # Wait 1 second until reconnection
done
17

It works with payload XXX but not with the original message:

{"child_lock":"UNLOCK","current_heating_setpoint":5,"deadzone_temperature":null,"heat":"OFF","linkquality":144,"local_temperature":16,"local_temperature_calibration":-5,"max_temperature":45,"max_temperature_limit":null,"min_temperature_limit":1,"preset":"hold","preset_mode":"hold","program":{"saturday_p1_hour":6,"saturday_p1_minute":0,"saturday_p1_temperature":5,"saturday_p2_hour":12,"saturday_p2_minute":0,"saturday_p2_temperature":5,"saturday_p3_hour":14,"saturday_p3_minute":0,"saturday_p3_temperature":34,"saturday_p4_hour":14,"saturday_p4_minute":8,"saturday_p4_temperature":5,"sunday_p1_hour":6,"sunday_p1_minute":0,"sunday_p1_temperature":5,"sunday_p2_hour":12,"sunday_p2_minute":30,"sunday_p2_temperature":5,"sunday_p3_hour":14,"sunday_p3_minute":0,"sunday_p3_temperature":34,"sunday_p4_hour":14,"sunday_p4_minute":8,"sunday_p4_temperature":5,"weekdays_p1_hour":6,"weekdays_p1_minute":0,"weekdays_p1_temperature":5,"weekdays_p2_hour":11,"weekdays_p2_minute":30,"weekdays_p2_temperature":5,"weekdays_p3_hour":18,"weekdays_p3_minute":0,"weekdays_p3_temperature":34,"weekdays_p4_hour":18,"weekdays_p4_minute":8,"weekdays_p4_temperature":5},"running_state":"idle","sensor":"AL","system_mode":"off"}

Tried to cut down the payload size with

msg=$(echo "$payload" | cut -d ' ' -f 2- | jq -j '.program |= {}' )

but no change; only the fixed string works.

18

PubSubClient just quietly stops (no error message or anything) receiving messages at length 243.

#!/bin/sh

n=1;
msg="x"

while [ $n -lt 230 ]; do
        msg=$( echo "x$msg" )
        n=$( echo "$n + 1" | bc )
done

while [ $n -lt 300 ]; do
        echo "Sending ${n}."
        mosquitto_pub -h localhost -t 'Esp32test' -m "$msg"
        msg=$( echo "x$msg" )
        n=$( echo "$n + 1" | bc )
        sleep 1
done
19

The max packet size defaults to 256. Try setBufferSize

20

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.