These days I tested how to integrate Arduino into MQTT, how to publish and how to subscribe to a topic.
For the most part, it's fine, I succeeded, but I'm facing a problem.
For example, in MQTT, the current hour, minute and second are published on a topic (with a frequency of once per second).
In arduino, due to the fact that the loop has a delay of, let's say 1.5 seconds or 2 seconds, when it reads the hour, minute and second from MQTT, at some point it ends up falling behind, because arduino seems to read all the old messages that were published on that topic, and not just the last message. As if these messages are somewhere in a queue, and the arduino reads them in turn.
If I stop the publication of the clock on that topic, the arduino reads all the messages until the moment I stop the publication. The behavior I want is for the arduino to read only the last message published there, and not the entire history, so the clock stays in sync. I need this functionality, not only for the clock, but also for other applications that require publishing data in mqtt very often, in which case I want the arduino to read my last message.
If I publish less often than the speed of the loop, the problem obviously disappears. But this is not a solution.
I tried to play with QOS or retain flag, but it doesn't solve the problem.
Is there a setting somewhere in the library where it can be set not to do message queing?
I can't because it has work to do, sensors to read, etc I am reading data from a solar inverter, and from 2 smart meters, and that requires time, can't do this without some delay...
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClient.h>
#include <PubSubClient.h>
const char* mqttServer = "x.x.x.x";
const int mqttPort = 1883;
const char* mqttUser = "xxxx";
const char* mqttPassword = "xxxxx";
const char* ssid = "xx";
const char* password = "xxxxx";
String serverName = "http://x.x.x.x/test/add_data.php?";
// the following variables are unsigned longs because the time, measured in
// milliseconds, will quickly become a bigger number than can be stored in an int.
unsigned long lastTime = 0;
// Timer set to 10 minutes (600000)
//unsigned long timerDelay = 600000;
// Set timer to 5 seconds (5000)
unsigned long timerDelay = 20000;
int var1, var2, var3, var4;
float fvar1;
int valint1, valint2, valint3;
float valfloat1, valfloat2, valfloat3;
int cday, chour, cminutes, cseconds;
int day_value, h_value, min_value, sec_value ;
//char cdy[8];
char ch[8];
char cmin[8];
char csec[8];
WiFiClient espClient;
PubSubClient client(espClient);
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);
Serial.println("Connecting");
while(WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.print("Connected to WiFi network with IP Address: ");
Serial.println(WiFi.localIP());
client.setServer(mqttServer, mqttPort);
client.setCallback(callback);
Serial.println("Connecting to MQTT...");
if (client.connect("ESP8266Client", mqttUser, mqttPassword )) {
Serial.println("connected");
} else {
Serial.print("failed with state ");
Serial.print(client.state());
delay(2000);
}
client.subscribe("homeassistant/datetime/day");
client.subscribe("homeassistant/datetime/hour");
client.subscribe("homeassistant/datetime/minutes");
client.subscribe("homeassistant/datetime/seconds");
}
void callback(char* topic, byte* payload, unsigned int length) {
String messageDay;
String messageHour;
String messageMinutes;
String messageSeconds;
if (strcmp(topic,"homeassistant/datetime/day") == 0){
for (int i=0;i<length;i++) {
messageDay += (char)payload[i];
}
day_value = messageDay.toInt();
}
if (strcmp(topic,"homeassistant/datetime/hour") == 0){
for (int i=0;i<length;i++) {
messageHour += (char)payload[i];
}
h_value = messageHour.toInt();
}
if (strcmp(topic,"homeassistant/datetime/minutes") == 0){
for (int i=0;i<length;i++) {
messageMinutes += (char)payload[i];
}
min_value = messageMinutes.toInt();
}
if (strcmp(topic,"homeassistant/datetime/seconds") == 0){
for (int i=0;i<length;i++) {
messageSeconds += (char)payload[i];
}
sec_value = messageSeconds.toInt();
}
}
void loop() {
transmitdata();
var1 = -1159;
fvar1 = 99545.2;
var2 = 35;
var3 = 45;
var4 = 55;
delay(1000); // this is where I induce the delay...
Serial.print("Current day is:");
Serial.println(day_value);
Serial.println("Current time is:");
Serial.print(h_value);
Serial.print(":");
Serial.print(min_value);
Serial.print(":");
Serial.print(sec_value);
client.loop();
}
void transmitdata() { // function that posts some data to mysql server
// Send an HTTP POST request depending on timerDelay
if ((millis() - lastTime) > timerDelay) {
//Check WiFi connection status
if(WiFi.status()== WL_CONNECTED){
WiFiClient client;
HTTPClient http;
String add_data = serverName;
add_data += "var1=";
add_data += var1;
add_data += "&";
add_data += "var2=";
add_data += var2;
add_data += "&";
add_data += "var3=";
add_data += var3;
add_data += "&";
add_data += "var4=";
add_data += var4;
// Your Domain name with URL path or IP address with path
http.begin(client, add_data.c_str());
// Send HTTP GET request
int httpResponseCode = http.GET();
// Free resources
http.end();
}
lastTime = millis();
}
}
I know delay is bad (but this is not the original code, I can't test the code live, that's why I induce the delay manually. Everytime I test something new, I test with a blank code and a new nodemcu to see how and if it works as desired. I know that is a delay, and can't do nothing about it, trust me. The original reads from 3 serials, one serial is the inverter, and the other 2 is a RS485 communication with 2 smart meters. There is no way I can read this instantaneously...
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClient.h>
#include <PubSubClient.h>
const char* mqttServer = "x.x.x.x";
const int mqttPort = 1883;
const char* mqttUser = "xxxx";
const char* mqttPassword = "xxxxx";
const char* ssid = "xx";
const char* password = "xxxxx";
String serverName = "http://x.x.x.x/test/add_data.php?";
// the following variables are unsigned longs because the time, measured in
// milliseconds, will quickly become a bigger number than can be stored in an int.
unsigned long lastTime = 0;
// Timer set to 10 minutes (600000)
//unsigned long timerDelay = 600000;
// Set timer to 5 seconds (5000)
unsigned long timerDelay = 20000;
unsigned long OneKDelay = 1000;
unsigned long PastOneKTime = millis();
int var1, var2, var3, var4;
float fvar1;
int valint1, valint2, valint3;
float valfloat1, valfloat2, valfloat3;
int cday, chour, cminutes, cseconds;
int day_value, h_value, min_value, sec_value ;
//char cdy[8];
char ch[8];
char cmin[8];
char csec[8];
WiFiClient espClient;
PubSubClient client(espClient);
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);
Serial.println("Connecting");
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.print("Connected to WiFi network with IP Address: ");
Serial.println(WiFi.localIP());
client.setServer(mqttServer, mqttPort);
client.setCallback(callback);
Serial.println("Connecting to MQTT...");
if (client.connect("ESP8266Client", mqttUser, mqttPassword )) {
Serial.println("connected");
} else {
Serial.print("failed with state ");
Serial.print(client.state());
delay(2000);
}
client.subscribe("homeassistant/datetime/day");
client.subscribe("homeassistant/datetime/hour");
client.subscribe("homeassistant/datetime/minutes");
client.subscribe("homeassistant/datetime/seconds");
}
void callback(char* topic, byte* payload, unsigned int length) {
String messageDay;
String messageHour;
String messageMinutes;
String messageSeconds;
if (strcmp(topic, "homeassistant/datetime/day") == 0) {
for (int i = 0; i < length; i++) {
messageDay += (char)payload[i];
}
day_value = messageDay.toInt();
}
if (strcmp(topic, "homeassistant/datetime/hour") == 0) {
for (int i = 0; i < length; i++) {
messageHour += (char)payload[i];
}
h_value = messageHour.toInt();
}
if (strcmp(topic, "homeassistant/datetime/minutes") == 0) {
for (int i = 0; i < length; i++) {
messageMinutes += (char)payload[i];
}
min_value = messageMinutes.toInt();
}
if (strcmp(topic, "homeassistant/datetime/seconds") == 0) {
for (int i = 0; i < length; i++) {
messageSeconds += (char)payload[i];
}
sec_value = messageSeconds.toInt();
}
}
void loop() {
transmitdata();
var1 = -1159;
fvar1 = 99545.2;
var2 = 35;
var3 = 45;
var4 = 55;
if ( (millis() - PastOneKTime) >= OneKDelay )
{
///delay(1000); // this is where I induce the delay...
Serial.print("Current day is:");
Serial.println(day_value);
Serial.println("Current time is:");
Serial.print(h_value);
Serial.print(":");
Serial.print(min_value);
Serial.print(":");
Serial.print(sec_value);
PastOneKTime = millis();
}
client.loop();
}
void transmitdata() { // function that posts some data to mysql server
// Send an HTTP POST request depending on timerDelay
if ((millis() - lastTime) > timerDelay) {
//Check WiFi connection status
if (WiFi.status() == WL_CONNECTED) {
WiFiClient client;
HTTPClient http;
String add_data = serverName;
add_data += "var1=";
add_data += var1;
add_data += "&";
add_data += "var2=";
add_data += var2;
add_data += "&";
add_data += "var3=";
add_data += var3;
add_data += "&";
add_data += "var4=";
add_data += var4;
// Your Domain name with URL path or IP address with path
http.begin(client, add_data.c_str());
// Send HTTP GET request
int httpResponseCode = http.GET();
// Free resources
http.end();
}
lastTime = millis();
}
}
that does not delay printing by one second?
Also doing a client loop without a WiFi connected test and a MQTT connected test is not wise.
void MQTTkeepalive( void *pvParameters )
{
sema_MQTT_KeepAlive = xSemaphoreCreateBinary();
xSemaphoreGive( sema_MQTT_KeepAlive ); // found keep alive can mess with a publish, stop keep alive during publish
// setting must be set before a mqtt connection is made
MQTTclient.setKeepAlive( 90 ); // setting keep alive to 90 seconds makes for a very reliable connection, must be set before the 1st connection is made.
for (;;)
{
//check for a is-connected and if the WiFi 'thinks' its connected, found checking on both is more realible than just a single check
if ( (wifiClient.connected()) && (WiFi.status() == WL_CONNECTED) )
{
xSemaphoreTake( sema_MQTT_KeepAlive, portMAX_DELAY ); // whiles MQTTlient.loop() is running no other mqtt operations should be in process
MQTTclient.loop();
xSemaphoreGive( sema_MQTT_KeepAlive );
}
else {
log_i( "MQTT keep alive found MQTT status %s WiFi status %s", String(wifiClient.connected()), String(WiFi.status()) );
if ( !(wifiClient.connected()) || !(WiFi.status() == WL_CONNECTED) )
{
connectToWiFi();
}
connectToMQTT();
}
vTaskDelay( 250 ); //task runs approx every 250 mS
}
vTaskDelete ( NULL );
}
Agreed. As far as I know, MQTT is "memoryless", messages from the broker to the client are not "queued". There is one exception ... messages published with the "retain" flag allow the last (and only last) message published to a topic to be downloaded when the client reconnects.
It's also not necessary to have your callback() function convert the c-string message to a String object just to then convert it to an integer. That's rather convoluted.
Why in the world are you doing that to begin with? The ESP8266 has a built-in RTC and access to NTP servers. It can always know the time without subscribing to an MQTT "Time' topic.
[/quote]
Why in the world are you doing that to begin with? The ESP8266 has a built-in RTC and access to NTP servers. It can always know the time without subscribing to an MQTT "Time' topic.
[/quote]
Ok. ESP has built-in RTC, but what about ArduinoMEGA2560 ?
As I sayd before, ignore the time example. let's pretend it's not the time in the topic, it's the available power from my solar panels to pass it directly and fast to the SSR of the boiler.
This value changes often and it must be read as fast as I can (and it should read the LAST message, not the whole history I consider that my question was not answered... why is this behaviour... thus I will make the suggested changes from the posts above, but I am afraid the problem is still there...
something like MQTT.fx would do the trick. MQTT.fx® 5.0 – Softblade and be a 3rd party confirmation that a MQTT Broker can now retain multiple messages.