Guidance on MQTT project

I have a hobby MQTT project that I am working on. I have just completed the hardware side of things, using a custom PCB that has each component integrated onto the board, so no breakout boards. MCU is an SMD esp32 and modem is an SMD quectel BG95, which uses a 4g LTE network. I use the arduino IDE on the ESP32.

I'd like to read a simple sensor on my board send it over cellular and then receive it on my webpage, Bluehost webserver. I have done HTTP posts from an ESP32 before to the website, but this gets complicated as you add more and more devices. So I am interested in trying MQTT, but don't really know how to get started. A few questions:

  1. Since my devices are using LTE network and webserver is not on my local network is there a good MQTT product/service I can use that won't break the bank?
  2. I'd like to use an RPi as the broker, but is this possible without opening ports on my router?

Other than that if anyone has general advice or suggestions I would love to hear.

Where is it? Can you have the MQTT server there too or elsewhere if the cloud?

Webserver is cloud-based I was planning on using this as my mqtt server as well though if I can figure it out.

Do you control that server? can you add packages? Bluehost probably has the capability, did you check their plans?

I think I should be able to I have MYSQL databases and have written code on it to receive HTTP posts and display values, which seems similar. But I don't know what all I need to do to get MQTT up and going on it. So I guess assume I can add packages

What type of hosting plan do you use at Bluehost?

I just have the "basic web hosting" plan

I don’t know their plans but from experience with other hosting companies, going beyond HTTP, PHP and MySQL is not in the basic plan, might need VPS hosting.

Do you mean "Broker"?

This may offer some help.

Thank you for the link. I think I did misspeak. I'd like an RPi in my local network to be the broker and the web-server and devices would be just clients to the broker. I should be able to make the web site a simple client. Although to make it act as a broker I don't think I have enough control over it as @J-M-L seems to be indicating.

Does anyone have any suggestions for this configuration

The Pi can easily run a website and an MQTT broker on your local network. If you want to view that website from outside, you'll need to open your firewall and deal with people trying to hack.

However, you could keep your website external and MQTT on the Pi and have some custom code on the Pi that acts as an MQTT client and forwards your sensor data to the external website. No need to open ports for this method.

1 Like

Thanks man this was helpful. I would prefer the later method since I am not as savvy with security protocol. I will be trying to set up SSL connections from my clients to the broker though for security.

Two questions here: Is this the same as the external website acting as an MQTT client to the broker? Secondly what about the fact that the sensors are on a cellular network, does this affect firewall/ports since the broker is on my local network?

No. The issue is that to be a client, the website would have to contact the broker, which again would mean poking a hole in the firewall. If the Pi contacts the website instead, it's more secure.

That may be more of a problem, but I'm no expert. How exactly does the data get to the website today?

Using HTTP protocol I can push data from my devices to the website. But once things go cellular data becomes very important, and the HTTP approach I had was very data heavy. No Pi Broker in this scenario.

I watched this video which accomplished something very similar: Mosquitto MQTT Android to Arduino

I had forgotten about the LTE transport. I was thinking that your sensor boards would be talking WiFi to something on your local net and then that data could be pushed to the external web site. MQTT would work well there.

But if you're sending data direct to the website via SMS, I don't see the benefit of adding a local broker. You could get it to work - for example you could have a master board that all the SMS goes to and then have it push sensor data into MQTT, but it's just extra complexity for not much I think.

Unless you have something local that's going to subscribe to the broker on the Pi, I'm not sure it'll help much apart from some learning.

It is HTTP not SMS. The point is that it'll use less data, but also as you mentioned for the learning. Learning is one of the main reasons I'd like to pursue MQTT. Also the MQTT method of publish subscribe is really easy to configure. When I used HTTP you have to setup the HTTP get and post commands custom for each device and it is easy for things to get messed up.

I have had multiple Arduinos and now a Pi or two reporting sensor data over HTTP. I used GET requests and gave each Arduino an ID in the query string, similarly each sensor it has has an ID too.

If you can get the sensor data to the Pi by some means, it can run a broker and echo the data to your cloud website as previously discussed. It might well be a valuable learning experience, but you may have to add some extra chicanery to get everything talking :thinking:

Sounds cool man. I know that it is possible and sounds like it functions pretty well for you, but because this is cellular HTTP will use more data than MQTT and cell data is expensive, and regardless I am just interested in giving MQTT a try.

I just found this link which had some great suggestions if you get a minute check it out and let me know what you think. MQTT mosquitto application

But instead of an RPi this guy used a VM so I don't think port forwarding was an issue

Why does it have to be by cellular? If you use the same public MQTT broker at each end then the cost would be minimal or even free. Or did I misunderstand your setup?

If the PI is in your home network and the sensor is not, and the sensor does not act as a server (hard with a basic cell connection) and if you want to use TCP-IP then you need to poke holes in your home network.

If you don’t want to poke holes, whatever is on your home network has to be a client of some server architecture hosted somewhere.

If you are willing to pay a bit of a monthly fee, then you can get the whole server stuff hosted.

1 Like