Arduino MQTT Client: dead messages

Hi. I have 6 test nodes up and running. My arduino MQTT client subscribes to my RAK7249 gateway. I receive messages from all 6 nodes. Goes well. However, I have turned one node off (node 9) and I have set node 5 to only send a packet when there has been a change from the last packet sent. There could be as long as 6 months between sending packets.

However, my Arduino MQTT Client regularly sees nodes 5 and 9 and reports messages from them. How can I stop these daed message, please.

Thank you

Are messages actually being sent or is the problem in the Arduino code ?

Perhaps but not limited to:

If the MQTT Broker has its persistent setting to true, then the last data sent will be available to any new client attaching. Persistence can be disabled. When persistence is disabled the MQTT Broker presents to its clients a NULL at initial connection. It is the client's responsibility to handle the reception of NULL payloads if persistence is disabled.

UKHeliBob

Here is the Arduino code -

// Arduino MQTT Client
//

#include <Ethernet.h>
#include <PubSubClient.h>
#include <SPI.h>
#include <ArduinoJson.h>
#include <String.h>
#include <lmic.h>
#include <stdio.h>
#include <base64.hpp>

StaticJsonDocument < 1024 > doc;
boolean TimeToParse = false;
char data[32] = "";
char devEUI[32] = "";
char applicationID[32] = "";

byte mac[] = { 0xA8, 0x61, 0x0A, 0xAE, 0x6A, 0x1A };                          //  MAC address Arduino Ethernet Shield MQTT client
IPAddress ip(192, 168, 1, 51);                                                //  Static IP address Arduino MQTT client
IPAddress server(192, 168, 1, 227);                                           //  Static IP address RAK7249 built-in LoRa server (Barn)
EthernetClient ethClient;
PubSubClient mqttClient(ethClient);
String clientID = String(mac[4]) + String(mac[5]) ;                           // use mac address to create clientID

//*******************************************************************************************************
// Callback
//*******************************************************************************************************

void receive(char * topic, byte * payload, unsigned int length)              //  ISR
{  
  deserializeJson(doc, payload, length);                                     // Deserialize the JSON document    
  TimeToParse = true;                                                        // set flag for loop to parse payload
  }

// ******************************************************************************************************
// Reconnect
//*******************************************************************************************************

void reconnect() 
{
  while (!mqttClient.connected())                                              // Loop until reconnected  
  {
    if (mqttClient.connect(clientID.c_str()))                                  // Attempt to connect
    {
      mqttClient.subscribe("application/+/device/+/rx");
    } 
    else 
    {
      delay(5000);                                                             // Wait 5 seconds before retrying
    }
  }
}

// ******************************************************************************************************
// setup
// ******************************************************************************************************

void setup() 
{
  Serial.begin(115200);
  Ethernet.begin(mac, ip);
  delay(5000);                                                                // Allow the hardware to sort itself out
  mqttClient.setServer(server, 1883);
  mqttClient.setCallback(receive);
}

// ******************************************************************************************************
// loop
// ******************************************************************************************************

void loop() 

{
  if (!mqttClient.connected()) 
  {
    reconnect();
  }
  mqttClient.loop();

  if (TimeToParse)                                                            // if true, then there's data to be parsed from the ISR
  {         
    strlcpy(applicationID, doc["applicationID"], sizeof(applicationID));      // parse payload for applicationID
    strlcpy(devEUI, doc["devEUI"], sizeof(devEUI));                           // parse payload for devEUI
    strlcpy(data, doc["data"], sizeof(data));                                 // parse payload for data
    
    {      
      Serial.print ("\napplicationID = ");
      Serial.print(applicationID);
      Serial.print ("  devEUI = ");
      Serial.print(devEUI);
      Serial.print("  data = ");
      Serial.print(data);
      Serial.print("\n");
    }
    

    TimeToParse = false;                                                      // turn off the flag
  }
}

Here is the output.

Node 5 is powered up and in service, but it only sends a packet when the mains/grid fails which tends to be about 6 times per year. The node reads the state of the grid every second, compared wirth the previous reading, and if changed, send s a messhe to thje gateway and then to the MQTT.

Node 9 is powered down just now. Not working, not in operatipn. not sending. However, it was going 9 days ago.

With regard to "presence", the RAK7249 dateway is set to Qos = 2.

Idahowalker

I've got a feeling that the two instances I ringed in red in the output are (i) the last data sent (albeit x days ago) when the MQTT is reset, ie initial connection, and (ii) the data associated with node 9 is a NULL while for node 5 (state of the grid) it is "7" representing 3 energised relays, ie bits 0,1,2 all set to 1.

I can solve node 9 by testing for the NULL. I can let node 5 transmit all the time ie every 1-second, and do the change of state testing in the loop of theMQTT. That will guarantee that there is a continuous connection to the gateway for that node, but will use up transmission time,

Before running client loop, I check for a valid WiFI connection first.
Using an ESP32, a client loop task.


void MQTTkeepalive( void *pvParameters )
{
  sema_MQTT_KeepAlive   = xSemaphoreCreateBinary();
  xSemaphoreGive( sema_MQTT_KeepAlive ); // found keep alive can mess with a publish, stop keep alive during publish
  // setting must be set before a mqtt connection is made
  MQTTclient.setKeepAlive( 90 ); // setting keep alive to 90 seconds makes for a very reliable connection, must be set before the 1st connection is made.
  for (;;)
  {
    //check for a is-connected and if the WiFi 'thinks' its connected, found checking on both is more realible than just a single check
    if ( (wifiClient.connected()) && (WiFi.status() == WL_CONNECTED) )
    {
      xSemaphoreTake( sema_MQTT_KeepAlive, portMAX_DELAY ); // whiles MQTTlient.loop() is running no other mqtt operations should be in process
      MQTTclient.loop();
      xSemaphoreGive( sema_MQTT_KeepAlive );
    }
    else {
      log_i( "MQTT keep alive found MQTT status %s WiFi status %s", String(wifiClient.connected()), String(WiFi.status()) );
      if ( !(wifiClient.connected()) || !(WiFi.status() == WL_CONNECTED) )
      {
        connectToWiFi();
      }
      connectToMQTT();
    }
    vTaskDelay( 250 ); //task runs approx every 250 mS
  }
  vTaskDelete ( NULL );
}
////

Note how the task starts out extending the MQTT keepalive value. The MQTT Broker likes to disconnect. It's the clients responsibility to maintain the connection.

Next, notice how a valid WiFi connection is checked, if ( (wifiClient.connected()) && (WiFi.status() == WL_CONNECTED) ) ? That is the most reliable way I've found to test for a valid WIFi connection.

Then finally client.loop() is requested.

If WiFi is found to be disconnected, then make a WiFi connection and a MQTT connection.

My code to make a WIFi connection


void connectToWiFi()
{
  int TryCount = 0;
  while ( WiFi.status() != WL_CONNECTED )
  {
    TryCount++;
    WiFi.disconnect();
    WiFi.begin( SSID, PASSWORD );
    vTaskDelay( 4000 );
    if ( TryCount == 10 )
    {
      ESP.restart();
    }
  }
  WiFi.onEvent( WiFiEvent );
} // void connectToWiFi()
////
void connectToMQTT()
{
  MQTTclient.setKeepAlive( 90 ); // needs be made before connecting
  byte mac[5];
  WiFi.macAddress(mac);
  String clientID = String(mac[0]) + String(mac[4]) ; // use mac address to create clientID
  while ( !MQTTclient.connected() )
  {
    // boolean connect(const char* id, const char* user, const char* pass, const char* willTopic, uint8_t willQos, boolean willRetain, const char* willMessage);
    MQTTclient.connect( clientID.c_str(), mqtt_username, mqtt_password, NULL , 1, true, NULL );
    vTaskDelay( 250 );
  }
  MQTTclient.setCallback( mqttCallback );
  MQTTclient.subscribe( topicOK );
} // void connectToMQTT()

Note the WiFi.disconnect(); use before making a WIFi connection? While WiFi.disconnect() does disconnect the WiFi, the important job of WiFi disconnect is to reset the WiFI stack.

Data persistence is a function of the MQTT Broker and not a function of the QoS setting.

Than you, all. I have been commissioning solar, so the mains has been on again, off again, many times in a day. I rebooted the RAK7249 gateway and problems went away, after I’d incorporated NULL messages into my code. Node 9 which is not powered up disappeared. Node 5 which monitors the solar/grid failure only shows up at MQTT start time. Thank you again.

1 Like

Hi again I have 3 nodes which the gateway tells me "are busy". The 4th only operates every 6 months so its not currently busy. However, the gateway says that I have 30 active nodes. I certainly haven't!

Can anyone explain why this is so, please. Usually the number of active nodes = the number of busy nodes. Could these be other farming users in the vicinity? Do they need to know my dateway EUI to come up on my screen?

What is a node?

I have a MQTT Broker and MQT clients. I have 13 MQTT clients. Each client sends and receives many payloads or just a few.

What is a node?

The MQTT Broker may be retaining security tokens for improperly disconnected devices. I'd not worry about it, except to make sure someone else is not using your broker without your knowledge.

This is not a MQTT Broker support forum, as a note.

What's a node?

Hi Idahowalker. I apologise if I'm on the wrong forum. I thought I was asking questions about software on an Arduino Mega. However, I take your point that most of what I have asked has been about MQTT Client. Can you please advise me where is the appropriate forum for this thread. Thank you.

Some Nodes are a LoRa Dragino shield mounted on an Arduino Mega. Another series of nodes is a RAK4630 in a Lora configuration. Each LoRa node has a sensor measuring attachments for example temperature and humditiy, PIR, straight out test node sending "Hello world", water level in tanks, state of solar electricity generation plant. The nodes send packets/messages to a centralised RAK7249 gateway (AU915 16 channel). The server associated with the gateway is accessed from the Arduino MQTT Client that has been the subject of this thread. It receives all the messages and present the data on a dashboard. In that way, I manage my farm.

Im hoping that I haven't taught you to suck eggs. My nodes are LoRa nodes, not MQTT clients.

However, having said that, I would not have got this project going with out your help, and for that I an very grateful. Kind regards

Hi, sorry, this isn't related to your thread, but I couldn't figure out how to send a private message on this forum.

I saw your old post and I am in the same situation as you with a RAK gateway with a built-in LoRa server. Could you share the sketch you used to send out your "Hello, Ryan" message?

I'm stuck at sending messages to the MQTT server using LoRa and I can't find any examples that aren't for connecting to the Things Network.

Thank you

Click on the poster's avatar and you will see an option to send a message to the user

Hi Mahalo. I’d be pleased to do that for you but I don’t know how to do that. I’m guessing that it is not appropriate to post code for you on this thread.

If you send ur email address to [email address removed], I will send you the code. Kind regards. John.

Moderator edit : email address removed as this is an open forum. Consider sending a PM instead

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.