ArduinoMqttClient Library - Question

Hi Forum :slight_smile:

I have a question regarding the ArduinoMqttClient Library. My project is about sending GPS Data from the Arduino MKR GSM 1400 to an MQTT client. When I’m publishing the data, I’m trying to set the boolean retain as true, but no matter what the message sent will always have retain set to false. However, for the will message, setting the retain boolean to true was no problem.

I then had a look at the code of the library and found this for the function beginMessage():

int beginMessage(const char* topic, unsigned long size, bool retain = false, uint8_t qos = 0, bool dup = false);
  int beginMessage(const String& topic, unsigned long size, bool retain = false, uint8_t qos = 0, bool dup = false);
  int beginMessage(const char* topic, bool retain = false, uint8_t qos = 0, bool dup = false);
  int beginMessage(const String& topic, bool retain = false, uint8_t qos = 0, bool dup = false);

here, it looks to me like the boolean is automatically set to false and what I’m trying to enter in my code will be overridden by this - am I understanding this correctly? This makes me wonder, if I’m wrong to try and set retain=true in the message I’m publishing? Can someone answer this for me?

I’m attaching a screenshot of my Serial Monitor (without the GPS Data), and my Code below:

#include <MKRGSM.h>
#include "arduino_secrets.h"
#include <ArduinoLowPower.h>
#include <Arduino_JSON.h>
#include <ArduinoMqttClient.h>

// PIN Number
const char PINNUMBER[]     = SECRET_PINNUMBER;
// APN data
const char GPRS_APN[]      = SECRET_GPRS_APN;
const char GPRS_LOGIN[]    = SECRET_GPRS_LOGIN;
const char GPRS_PASSWORD[] = SECRET_GPRS_PASSWORD;

// initialize the library instance
GSMClient client;
GPRS gprs;
GSM gsmAccess;
GSMLocation location;

MqttClient mqttClient(client);

//MQTT Broker Settings
char mqttServerAddress[] = "m24.cloudmqtt.com";
int mqttServerPort = 10334;
char mqttServerUsername[] = SECRET_MQTT_SERVER_USERNAME;
char mqttServerPassword[] = SECRET_MQTT_SERVER_PASSWORD;

const char willTopic[] = "TrashCollector/Gotham/will";
const char inTopic[]   = "TrashCollector/Gotham/#";
const char outTopic[]  = "TrashCollector/Gotham/010";

// define pin
const int buttonPin = 3;

// define variables
int buttonState = 0;
int collected = 0;
int intervalSend = 0;
const char cityIdentifier[] = "Gotham";
const char binNo[] = "010";

void setup() {
  // Initialize serial and wait for port to open:
  Serial.begin(9600);

  pinMode(buttonPin, INPUT);    // initialize the button pin as an input
  collected = 0;
  Serial.println("Starting Arduino GPRS ping.");
  // connection state
  bool connected = false;

  // After starting the modem with GSM.begin()
  // attach the shield to the GPRS network with the APN, login and password
  while (!connected) {
    if ((gsmAccess.begin(PINNUMBER) == GSM_READY) &&
        (gprs.attachGPRS(GPRS_APN, GPRS_LOGIN, GPRS_PASSWORD) == GPRS_READY)) {
      connected = true;
    } else {
      delay(1000);
    }
  }
  location.begin();

  Serial.print("\n***Starting connection to MQTT broker: ");
  Serial.println(mqttServerAddress);

  mqttClient.setUsernamePassword(mqttServerUsername, mqttServerPassword);
  mqttClient.connect(mqttServerAddress, mqttServerPort);

  if (!mqttClient.connect(mqttServerAddress, mqttServerPort)) {
    Serial.print("MQTT connection failed! Error code = ");
    Serial.println(mqttClient.connectError());
    reconnect();
  } else {
    Serial.println("Connection to MQTT broker set up");
  }

  // set the message receive callback
  mqttClient.onMessage(onMqttMessage);

  // set a will message
  String willPayload = "oh no!";
  bool willRetain = true;
  int willQos = 1;

  mqttClient.beginWill(willTopic, willPayload.length(), willRetain, willQos);
  mqttClient.print(willPayload);
  mqttClient.endWill();

  // subscribe to a topic
  // the second paramter set's the QoS of the subscription,
  // the the library supports subscribing at QoS 0, 1, or 2
  int subscribeQos = 1;

  mqttClient.subscribe(inTopic, subscribeQos);
}

void loop() {

  // call poll() regularly to allow the library to receive MQTT messages and
  // send MQTT keep alives which avoids being disconnected by the broker
  mqttClient.poll();

  // read the state of the pushbutton value:
  buttonState = digitalRead(buttonPin);
  if (buttonState == HIGH && collected == 0) {
    //trash has not been collected, the user wants to publish collection request
    sendUpdate(collected);
    collected = 255;
    //delay(1000);
  } else if (buttonState == HIGH && collected == 255) {
    //trash is being collected
    sendUpdate(collected);
    collected = 0;
  } 
}

void sendUpdate(int collected) {
  //String message = createJSON(collectME);
  JSONVar messageObject;

  bool collectedBool = collected;
  Serial.println(collectedBool);
  Serial.println(collected);

  //create JSON Object
  messageObject["cityIdentifier"] = cityIdentifier;
  messageObject["binNo"] = binNo;
  messageObject["latitude"] = location.latitude();
  messageObject["longitude"] = location.longitude();
  messageObject["collected"] = collectedBool;

  //Turn JSON Object into String
  String message = JSON.stringify(messageObject);

  Serial.print("JSON.stringify(message) = ");
  Serial.println(message);
  Serial.println();

  bool retained = true;
  int qos = 1;
  bool dup = false;

  mqttClient.beginMessage(outTopic, message.length(), retained, qos, dup);
  mqttClient.print(message);
  mqttClient.endMessage();

}

void onMqttMessage(int messageSize) {
//shortened so it fits on forum, can be seen on the library docu page
}

// reconnect logic for MQTT
void reconnect() {
  //shortened
}

here, it looks to me like the boolean is automatically set to false

If you don't supply a value for that position in the argument list, that is what happens. You can supply a different value.

What do you expect setting retain to true to accomplish?

here, it looks to me like the boolean is automatically set to false and what I'm trying to enter in my code will be overridden by this - am I understanding this correctly?

No, in the header just the default value is set. So if you don't specify a value for "retain", false will be used.

When I'm publishing the data, I'm trying to set the boolean retain as true, but no matter what the message sent will always have retain set to false. However, for the will message, setting the retain boolean to true was no problem.

I get the impression you have a wrong expectation of what the retain flag does. If you publish a message with the retain flag, the server stores the message. If a client subscribes to that topic, the retained message will immediately be sent (and the retained flag is set). But if later in the subscription another publish occurs the subscriber gets the new message but the retained flags will be clear (although the published did set it) because the server did not send a stored (old) message but a message it received that moment.

Hi - thanks pylon and paul for your replies! What you're saying about the library makes sense.

I want to set retain as true on the published messages so that a new subscriber can get the last message. I want to connect it to an app (the subscriber) and I want to make sure the data is received later even if the app is not connected at first. However, from what I can see on the MQTT toolbox, the published messages are never sent with retain set to true, even though I do that in the code.

pylon:
But if later in the subscription another publish occurs the subscriber gets the new message but the retained flags will be clear (although the published did set it) because the server did not send a stored (old) message but a message it received that moment.

Can you explain this in different words? I'm not sure I fully understand.. Are you saying that whether or not the retain flag is shown as true depends on whether the subscriber gets it as a retained message? So, it will always show the message with retain = false if the subscriber is connected in the moment the message is published? :o that seems very confusing.. especially since the other variables (QoS and duplicate) appear the way they've been set..

However, from what I can see on the MQTT toolbox, the published messages are never sent with retain set to true, even though I do that in the code.

I guess you don't know how to use that tool. Disconnect the MQTT toolbox, start your Arduino, then connect your MQTT toolbox and set the subscription. You will immediately get a message and that one has it's retained flag set. Am I right?

that seems very confusing.. especially since the other variables (QoS and duplicate) appear the way they've been set..

If the duplicate flag is copied from the publish message, your broker does something wrong. The flag is used to indicate if the sending party got the message acknowledged or not. If the broker got a message with the duplicate flag set and it copies that to the outgoing message it sends for the first time, this is an implementation bug.
If the client sends a retain flag in a publish message to the broker, the broker stores the last received publish message for that topic and sends it immediately to new topic subscribers. If the new subscriber got a stored message, it's retain flag will be set. If a client sends a publish message to the broker with the retain flag set for a given topic and that topic has an active subscriber at that moment, the subscriber will get that message but the retain flag will not be set because the message is current and not a stored information.