MQTT/IoT (general) questions...

Other Hardware General Electronics
1

I have played with the ESP8266 modules before..

  • captive portals (mostly used as GUI's to control/interact with the connect project
  • reach out and dump data to some custom PHP script (hosted on my own domain/server)..

but I have never used any of these IoT 'services' to be honest.

I keep seeing the 'buzz word' MQTT.. etc..etc.. yadda yadda..

but I'm having troubling grasping some things about it.. and am hoping some conversation clears things up for me.

I understand that MQTT is a publish/subscribe messaging protocol...

but is this a constantly connected/polling type of connection? So you are always using bandwidth, needing to be connected to be able to 'get' the response/pushed data..... is this correct?

What is the difference between this and having some script on the internet and you request every so often?

  • I read this MQTT protocol was designed to be lightweight... but with todays bandwidth/connections is that an issue any more? (maybe with multiples or professional grade stuff?)

** I read it also has more offerings like LWT (Last Will and Testament) messages? So seems a bit more robust than a simple script that gets polled every so often.

Lastly.. is it possible to set-up/install your own MQTT server?

I see things like Mosquito and mqtt-server... but have read much about them yet.

Maybe possible to install ona home computer/server that gets opened up to the internet via router access or something? but what about shared domain hosting? (and thoughts/feedback on that?)

I even saw some articles about installing it on a RaspPI??? (that would be fun first RaspPi project for me IMO!)

thanks!

2

I keep seeing the 'buzz word' MQTT.. etc..etc.. yadda yadda..

It's not Buzzword, it is actually pretty awesome.

but is this a constantly connected/polling type of connection? So you are always using bandwidth, needing to be connected to be able to 'get' the response/pushed data..... is this correct?

As far as I know, it does try to keep a TCP connection open, yes. Traffic is little, though.

What is the difference between this and having some script on the internet and you request every so often?

  • You don't have to go through a lengthy HTTP dialog.
  • Entities try to keep connected, so you don't have to authenticate and poll all the time
  • It is super fast. Instant enough to live-control a dimmer by sending increase/decrease messages to to a server that the dimmer is subscribed to, without overshooting all the time.
  • You have QoS (not all supported by the MCU lib, but in general, you can make sure that messages arrive at least or exactly once)
  • Other entities can subscribe without having to change the server
  • You have LWT, that is a message that is posted when the server loses connection (and timeouts for reconnects) to the node.
  • I read this MQTT protocol was designed to be lightweight... but with todays bandwidth/connections is that an issue any more? (maybe with multiples or professional grade stuff?)

Depends. MQTT can push it's message through a 16kbit/s throttled mobile connection. I wouldn't want to do that with HTTP. Also, your MCU has much less to do compared to polling via HTTP, authenticating, parsing ...

Lastly.. is it possible to set-up/install your own MQTT server?

Sure. Mosquitto is a very easy to setup server. It is in the Debian/Ubuntu repositories.

Maybe possible to install ona home computer/server that gets opened up to the internet via router access or something?

Sure, Dynamic DNS is your friend.

but what about shared domain hosting? (and thoughts/feedback on that?)

No, it is not a PHP script, it is actual compiled software. You need root access to at least a vserver.

I even saw some articles about installing it on a RaspPI??? (that would be fun first RaspPi project for me IMO!)

Sure, no problem.
sudo apt-get install mosquitto(-server?)

Anyway MQTT is the best thing since sliced bread. I use it for

  • All home automation communication, including switches, sensors ... with OpenHAB as one end and Wemos D1 Minis (ESP8266) as most of the others
  • Mobile location tracing with Owntracks

Recently, I build an MQTT to IR converter. Reads IR and dumps the info (protocol, bits, address, code) to MQTT topics, and subscribes to other topics and sends received info as IR. It is just great to have a unified place where you can dump all that communication from wireless units in an ordered way.

1 Like
3

Thanks for the reply.

This was in my email this morning. (coincidence?)

but I thought gave a decent over-view:

4

Looks like a nice high level overview.
As a practical addition:
I like to have two topics for a list:
home/firstfloor/bedroom/indirect/0/state
home/firstfloor/bedroom/indirect/0/cmd

In the latter topic, other instances can send commands: INCREASE, DECREASE, [percent], ON, OFF
In the first topic, the lamp publishes it's current state, no matter where it came from. This might be from:

  • MQTT
  • Directly connected buttons
  • Sensors with an internally programmed connection to the light
  • A dimmer (I kept a few PWM remote dimmer controls and just read their PWM signal with the Arduino)
  • ...

You can also implement online/offline modes so that the buttons run through Openhab rules when the server is available, and do their thing locally if not.
Anyway, that division gives you more flexibility while the Openhab server still knows the state at all times.

5

IMHO the important aspect with MQTT is that a device ("publisher" or "client") establishes relationships with the brokers connecting to it. So if you want to make a doorbell device, for example, with MQTT you can ensure that when someone presses the doorbell that all of the brokers interested in that event get the message right away. The broker doesn't have to constantly poll the doorbell to watch for changes in state. Then to answer your question about persistent connections between brokers/clients, no, they don't have to be, but it is preferred that they are for situations like a doorbell.

6

el97, you posted the exact questions I have had for a couple weeks now. I've been using Mosquitto on a RPi with EmonCMS (though without really understanding how MQTT works) and it has been remarkable and amazingly reliable. Now I'm on the quest to understand it more, so this thread is very relevant.

So, with PubSubClient, I see there's a keep-alive defaulted to 15 seconds. It sounds like a big difference between HTTP and MQTT is that HTTP needs to re-establish a connection each time, while MQTT keeps one open. My impression is that this is what Chagrin is referring to by the established relationships. Is this on track?

7

I believe that is correct (re: Chagrin comment)

  • HTTP will need to connect each time..

And that was sort of my point in asking..

What is the difference between this MQTT protocol and just polling some webservice/script..etc

not having to deal with HTTP/headers..etc is nice.. but I wasnt sure if that really 'matters' in this day and age of bandwidth..etc

If MQTT keeps a constant connection.. thats nice.. but I (again) wasnt clear on how much overhead this really adds? and (again) is the a PRO to having a 'constant' connection vs. polling when you went (or a timed polling even)

When connected to MQTT server.. (and nothing is going on).. are you burning up bandwidth at all?

If I can install this on a RPi.. and then open my router so this will be available to me wherever I am/go.. I see that a plus for me.

Internally (like a home automated network only, no access to outside).. I'm not sure I see much of an advantage yet..

I have never worked with the Rpi before.. and now all in one week.. I have 3 RPi based projects I want to do! HAHA..

(thanks ADHD!) :stuck_out_tongue_winking_eye:

Anyone is the know care to comment on the differences between RED Node and Mosquito?

8

I'm newish to both, but recently set up a Raspberry Pi with EmonCMS (which is awesome), which uses Node-Red and Mosquitto. I have an Arduino running PubSubClient to post electricity readings, and a bunch of ESP8266s running PubSubClient posting temp/humidity readings via MQTT to the Pi (Mosquitto). Apparently EmonCMS is a subscriber to those queues. I have been fascinated by how well this has worked with all kinds of things being thrown at it. (and the simplicity of the code to do so)

They are really two very different things. I have played much more with Node-Red. Node-Red is an integration masterpiece that makes it easy to integrate virtually anything. I'm using it to pass arguments to a Python script that communicates with an Arduino via I2C to drive my home office lighting (which is done with Neopixels).

9

I've found MQTT to be very reliable for my uses.. I use it to send inputs to from a touchscreen wirelessly to a controller that controls WS2801's. Very fast.

The only issue is for me is that if the MQTT server goes down for whatever reason, devices attempting to reconnect to it spend a lot of time in the reconnection attempts and it basically locks up the arduino/esp8266 for a while until reconnection is restablished... But the only time i've had the server go out was before I set it up to use a static IP.... I am using an old android tablet as my MQTT server. I wish there was a way to use an ESP8266 as a server but there doesn't seem to be anything available.

10

As per my understanding, In a nut shell this is what happens

  1. MQTT client subscribe to a topic by calling the server.
  2. The server registers the client and the topic.

After the above steps there is no bandwidth used.

When some one publish a message to a topic, server pushes this details to all the client listening to that topic. Again bandwidth used here.

I wrote a small post about MQTT communication with ESP, check this post MQTT Communication with Arduino using ESP8266 ESP-01 | Sony Arouje

I also posted a nodejs MQTT server.

For testing an MQTT server you can use heroku and use third party addons for MQTT.

11

To give you an idea, my phone is connected 24/7 with owntracks, reported about 500 positions and says Owntracks used about 5MB of traffic.