How to handle mqttclient and http post in loop

Using Arduino Programming Questions
1

Scenario:
Existing project: read soft serial then do http post
New project: MQTT player from other (MrDIY / MrDIY Audio Notifier · GitLab)

Objective: Combine together (Result: successful )

Issue: When read serial do http post, after http.end(). MQTT client disconnect.
It taks above 10 secs to reconnect.

The only way I found is just a workaround to add ESP.restart(); after each http post.
But that's stupid. I have spend more than 10 hours to fix this issue.
Any one can help. Apperaciate and great thanks.

Eg code:

void loop() {
  if (!mqttClient.connected()) {
    mqttReconnect();
  }
  mqttClient.loop();

  if (mySerial.available() > 0)  //Checks is there any data in buffer
  {
    receive_data = String(receive_data) + String(char(mySerial.read()));
    //Serial.println(receive_data);

    if (String(receive_data).indexOf("lighton") >= 0) {
      Serial.println("lighton");
      receive_data = "";
      http.begin(client, switchon);
      http.addHeader("Content-Type", "application/json");
      http.addHeader("Authorization", auth);
      int httpCode = http.POST("{\"entity_id\": \"switch.sonoff_guest_room\"}");
      Serial.printf("[http] GET... code: %d\n", httpCode);
      http.end();
      //ESP.restart(); 
    }
}
2

Could you post the entire code?

3

Sure. Please help to check


#include "Arduino.h"
#include "boot_sound.h"
#include "ESP8266WiFi.h"
#include "PubSubClient.h"
#include "AudioFileSourceHTTPStream.h"
#include "AudioFileSourceICYStream.h"
#include "AudioFileSourcePROGMEM.h"
#include "AudioFileSourceBuffer.h"
#include "AudioGeneratorMP3.h"
#include "AudioGeneratorWAV.h"
#include "AudioGeneratorRTTTL.h"
#include "AudioOutputI2SNoDAC.h"
#include "ESP8266SAM.h"


#include <ESP8266HTTPClient.h>
#include <SoftwareSerial.h>

SoftwareSerial mySerial(13, 15);//(D7 RX,D8 TX  )
const char* switchoff = "http://xxxxxx";
const char* switchon = "http://xxxxxxx";
const char* auth = "Bearer xxxxxxx";
String receive_data;

AudioGeneratorMP3         *mp3 = NULL;
AudioGeneratorWAV         *wav = NULL;
AudioGeneratorRTTTL       *rtttl = NULL;
AudioFileSourceHTTPStream *file_http = NULL;
AudioFileSourcePROGMEM    *file_progmem = NULL;
AudioFileSourceICYStream  *file_icy = NULL;
AudioFileSourceBuffer     *buff = NULL;
#ifdef USE_I2S
AudioOutputI2S            *out = NULL;
#else
AudioOutputI2SNoDAC       *out = NULL;
#endif

WiFiClient                client;
PubSubClient          mqttClient(client);
HTTPClient               http;

#define  MQTT_MSG_SIZE    256
const char* willTopic     = "LWT";
const char* willMessage   = "offline";
boolean willRetain        = false;
byte willQoS              = 0;

// AudioRelated ---------------------------
float volume_level              = 0.8;
String playing_status;
const int preallocateBufferSize = 2048;
void *preallocateBuffer = NULL;

// WifiManager -----------------------------
// WiFi
const char *ssid = "ChaCha2.4"; // Enter your WiFi name
const char *password = "xxxxx";  // Enter WiFi password

// MQTT Broker
const char *mqttServer = "192.168.0.200";
const char *mqttUserName = "xxx";
const char *mqttUserPassword = "xxx";
const char *mqttTopicPrefix = "speaker";
char mqttTopic[MQTT_MSG_SIZE];


/* ################################## Setup ############################################# */

void setup() {


  Serial.begin(115200);
  mySerial.begin(115200); //opens soft serial port
  wifiConnected();
  mqttClient.setServer(mqttServer, 1883);
  mqttClient.setCallback(onMqttMessage);
  mqttClient.setBufferSize(MQTT_MSG_SIZE);
  out = new AudioOutputI2SNoDAC();
  Serial.println("Using No DAC - using Serial port Rx pin");
  out->SetGain(volume_level);
}

/* ##################################### Loop ############################################# */

void loop() {


  if (!mqttClient.connected()) {
    mqttReconnect();
  }

  mqttClient.loop();
  if (mySerial.available() > 0)  //Checks is there any data in buffer
  {
    receive_data = String(receive_data) + String(char(mySerial.read()));

    //Serial.println(receive_data);
    if (String(receive_data).indexOf("lighton") >= 0) {

      Serial.println("lighton");
      receive_data = "";
      http.begin(client, switchon);
      http.addHeader("Content-Type", "application/json");
      http.addHeader("Authorization", auth);
      int httpCode = http.POST("{\"entity_id\": \"switch.sonoff_guest_room\"}");
      Serial.printf("[http] GET... code: %d\n", httpCode);
      http.end();
      //ESP.restart();

    } 
  }

  if (mp3   && !mp3->loop())    stopPlaying();
  if (wav   && !wav->loop())    stopPlaying();
  if (rtttl && !rtttl->loop())  stopPlaying();

  //#ifdef DEBUG_FLAG
  //  if (mp3 && mp3->isRunning()) {
  //    static int unsigned long lastms = 0;
  //    if (millis() - lastms > 1000) {
  //      lastms = millis();
  //      Serial.print(F("Free: "));
  //      Serial.print(ESP.getFreeHeap(), DEC);
  //      Serial.print(F("  ("));
  //      Serial.print( ESP.getFreeHeap() - ESP.getMaxFreeBlockSize(), DEC);
  //      Serial.print(F(" lost)"));
  //      Serial.print(F("  Fragmentation: "));
  //      Serial.print(ESP.getHeapFragmentation(), DEC);
  //      Serial.print(F("%"));
  //      if ( ESP.getHeapFragmentation() > 40) Serial.print(F("  ----------- "));
  //      Serial.println();
  //    }
  //  }
  //#endif
}

/* ############################### Audio ############################################ */

void playBootSound() {
  Serial.println("Start playBootSound");
  file_progmem = new AudioFileSourcePROGMEM(boot_sound, sizeof(boot_sound));
  wav = new AudioGeneratorWAV();
  wav->begin(file_progmem, out);
}

void stopPlaying() {
  Serial.println("Start stopPlaying");
  if (mp3) {
#ifdef DEBUG_FLAG
    Serial.print(F("...#"));
    Serial.println(F("Interrupted!"));
    Serial.println();
#endif
    mp3->stop();
    delete mp3;
    mp3 = NULL;
  }
  if (wav) {
    wav->stop();
    delete wav;
    wav = NULL;
  }
  if (rtttl) {
    rtttl->stop();
    delete rtttl;
    rtttl = NULL;
  }
  if (buff) {
    buff->close();
    delete buff;
    buff = NULL;
  }
  if (file_http) {
    file_http->close();
    delete file_http;
    file_http = NULL;
  }
  if (file_progmem) {
    file_progmem->close();
    delete file_progmem;
    file_progmem = NULL;
  }
  if (file_icy) {
    file_icy->close();
    delete file_icy;
    file_icy = NULL;
  }
  broadcastStatus("status", "idle");

}

/* ################################## MQTT ############################################### */

void onMqttMessage(char* topic, byte* payload, unsigned int mlength)  {
  Serial.println("Start onMqttMessage");
  char newMsg[MQTT_MSG_SIZE];

  if (mlength > 0) {
    memset(newMsg, '\0' , sizeof(newMsg));
    memcpy(newMsg, payload, mlength);
#ifdef DEBUG_FLAG
    Serial.println();
    Serial.print(F("Requested ["));
    Serial.print(topic);
    Serial.print(F("] "));
    Serial.println(newMsg);
#endif
    // got a new URL to play ------------------------------------------------
    if ( !strcmp(topic, mqttFullTopic("play") ) ) {
      stopPlaying();
      file_http = new AudioFileSourceHTTPStream();
      if ( file_http->open(newMsg)) {
        broadcastStatus("status", "playing");

        buff = new AudioFileSourceBuffer(file_http, preallocateBuffer, preallocateBufferSize);
        mp3 = new AudioGeneratorMP3();
        mp3->begin(buff, out);
      } else {
        stopPlaying();
        broadcastStatus("status", "error");
        broadcastStatus("status", "idle");
      }
    }

    // got a new URL to play ------------------------------------------------
    if ( !strcmp(topic, mqttFullTopic("stream"))) {
      stopPlaying();
      file_icy = new AudioFileSourceICYStream();
      if ( file_icy->open(newMsg)) {
        broadcastStatus("status", "playing");
        buff = new AudioFileSourceBuffer(file_icy, preallocateBuffer, preallocateBufferSize);
        mp3 = new AudioGeneratorMP3();
        mp3->begin(buff, out);
      } else {
        stopPlaying();
        broadcastStatus("status", "error");
        broadcastStatus("status", "idle");
      }
    }

    // got a tone request --------------------------------------------------
    if ( !strcmp(topic, mqttFullTopic("tone") ) ) {
      stopPlaying();
      broadcastStatus("status", "playing");
      file_progmem = new AudioFileSourcePROGMEM( newMsg, sizeof(newMsg) );
      rtttl = new AudioGeneratorRTTTL();
      rtttl->begin(file_progmem, out);
      broadcastStatus("status", "idle");
    }

    //got a TTS request ----------------------------------------------------
    if ( !strcmp(topic, mqttFullTopic("say"))) {
      stopPlaying();
      broadcastStatus("status", "playing");
      ESP8266SAM *sam = new ESP8266SAM;
      sam->Say(out, newMsg);
      delete sam;
      stopPlaying();
      broadcastStatus("status", "idle");
    }

    // got a volume request, expecting double [0.0,1.0] ---------------------
    if ( !strcmp(topic, mqttFullTopic("volume"))) {
      volume_level = atof(newMsg);
      if ( volume_level < 0.0 ) volume_level = 0;
      if ( volume_level > 1.0 ) volume_level = 0.7;
      out->SetGain(volume_level);
    }

    // got a stop request  --------------------------------------------------
    if ( !strcmp(topic, mqttFullTopic("stop"))) {
      stopPlaying();
    }
  }
}

void broadcastStatus(const char topic[], String msg) {
  Serial.println("Start broadcastStatus");
  if ( playing_status != msg) {
    char charBuf[msg.length() + 1];
    msg.toCharArray(charBuf, msg.length() + 1);
    mqttClient.publish(mqttFullTopic(topic), charBuf);
    playing_status = msg;

  }
}


void mqttReconnect() {
  Serial.println("Start mqttReconnect");
  while (!mqttClient.connected()) {
    String clientId = "DIYSpeaker-";

    // Attempt to connect

    if (mqttClient.connect(clientId.c_str(), mqttUserName, mqttUserPassword, mqttFullTopic(willTopic), willQoS, willRetain, willMessage)) {
      mqttClient.subscribe(mqttFullTopic("play"));
      mqttClient.subscribe(mqttFullTopic("stream"));
      mqttClient.subscribe(mqttFullTopic("tone"));
      mqttClient.subscribe(mqttFullTopic("say"));
      mqttClient.subscribe(mqttFullTopic("stop"));
      mqttClient.subscribe(mqttFullTopic("volume"));

      Serial.println(F("Connected to MQTT"));
      playBootSound();
      broadcastStatus("LWT", "online");
      broadcastStatus("ThingName", clientId.c_str());
      broadcastStatus("IPAddress", WiFi.localIP().toString());
      broadcastStatus("status", "idle");
    }
  }
  Serial.println("End mqttReconnect");
}
/* ############################ WifiManager ############################################# */

void wifiConnected() {

  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  randomSeed(micros());

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

}


char* mqttFullTopic(const char action[]) {
  strcpy (mqttTopic, mqttTopicPrefix);
  strcat (mqttTopic, "/");
  strcat (mqttTopic, action);
  return mqttTopic;
}
4

My first suggestion is to use String a bit differently. Instead of a String variable that can consume stack space and make memory holes. Yes, I know the ESP8266 has a lot of RAM and if the ESP8266 has freeRTOS Strings are not a big issue but. Ah. Anyways in setup () reserve a space for your string by setting a memory space for the String to use with receive_data.reserve( XX ) where XX is at least 1 character larger then the largest sized string to be held in the buffer. Before making a buffer, use receive_data.lentgh() in a print to get the various sizes, picking out the largest to use in the buffer.

OK. To use the String set as a buffer receive_data.concat( "dfjvn" ) adds to the string buffer. The concat() adds the String to the end of the buffer. To clear the buffer and set the buffer for the next string operation use receive_data = "";

Look into the PubSubClient library documentation ( Arduino Client for MQTT (knolleary.net)) to see how to set the default disconnect timeout to more than a few seconds. I think the max value is 90 seconds. It's in the docs. Oh, the MQTT Broker really likes to disconnect its clients. ( setKeepAlive())

EspSoftwareSerial - Arduino Reference

5

Thanks for the reply.
I quite don't understand why the receive_date will affect
when mySerial.available() > 0,
running http.begin
running http.end
Then cause mqtt client disconnect.

Because if I have no byte coming from soft serial. The mqtt client will run without any issue. Never disconnect.

And I just test if I comment the code between http.begin until http.end.
Serial read without any issue. MQTT will not disconnect.
So the main problem is http.end also disconneted MQTT connection.

6

Related topic found

7

Issue resolved by define HTTPClient http; in the loop instead of define in global. MQTT client still will be disconnect after http.end but it will reconnect in 1 sec.

8

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.