I'm using the ESP32 with the Arduino IDE. Currently, my ESP32 is functioning as a Web Server thanks to the WiFiServer.h in Arduino's Wifi library. I have a a trigger setup on my remote database (I'm using 8base) that fires whenever a new record added to the table. This trigger uses an API call to post some data to the ESP32 server.
Right now, I'm using ngrok to expose my ESP32 to the internet. This works for development, but, eventually, I'd like to have multiple ESP32s at different locations that may be on separate WiFi networks. From what I understand, if I wanted to ngrok for these ESP32s, I'd need to purchase a separate ngrok tunnel for each location, which doesn't seem very cost efficient.
From doing a bit of research, it seems like one option is to expose a specific port on each network where my ESP32s are deployed. However, I've used other microcontrollers, such as Particle Boron devices, where this additional setup isn't necessary. With Particle, there's a Particle platform that abstracts away a lot of what's going on "under the hood", but that system functions by allowing user's to define "Cloud Functions", which essentially expose an individual function on the microcontroller's firmware through a REST API endpoint. I found a platform for ESP32 called Rainmaker that seems similar; although, it comes with a pretty steep price tag. I've seen some forum posts where developers are using MQTT to accomplish similar things. It seems like all I'd need for this is a broker like HiveMQ, and I'd be able to use my devices without setting up special network rules. My guess is that this is what Particle is using behind the scenes to make their service work.
My question is: Is my assessment of my alternatives correct, or is there something I'm missing? It seems like I have three options:
Manually expose a port at each site for my ESP32 devices and run them as servers
Pay for a service like Rainmaker to mange this for me with APIs that abstract away the complexity of this.
Setup an MQTT network using a public broker.
Any insight from folks who are more experienced with networking than me would be greatly appreciated!
My rule of thumb is to never have to open a personal/home network to random incoming traffic as this is a significant increase the attack surface of your personal network and thus should be avoided.
So if the ESP sits inside the home network and is not reachable from the outside through an IP request ➜ it means the ESP itself has to go fetch its information (possibly maintaining an open connection with a server) and the ESP has the initiative to open the connection.
The server itself needs to be reachable from anywhere. My choice is to pay for a hosting service for a virtual private server (VPS) and I can run whatever I want there (MQQT, databases, webserver, FTP.... but also my own PHP or Python scripts etc).
if paying for the service is not an option, then having a RPi in the DMZ of your home network and making sure you have a good firewall etc can also be an option.
(I've 2 separate internet access at home and one is dedicated to tinkering and personal projects with no sensitive personal information and I've an RPi in the DMZ. if that network were to be compromised that would not be a catastrophe).
Do you need a secure/encrypted connection that can reliably handle hundreds or thousands of messages a day?
If not then just setup a simple MQTT publishing demo to test.mosquitto.org using long random topic names so you don't get collisions with the other traffic.
If you want to keep using HTTP, rather than switching to something like MQTT, you could pivot your architecture a bit: have a central server (in your home network if you're comfortable with that, on a vserver or in the cloud if not), have your ESP32 devices regularly push their data to this server via HTTP, and then do whatever presentational things you want to do on the central server.
That way, the ESP32 only ever acts as a HTTP client and doesn't need any port exposures.
If you control the networks these devices will be deployed to, and know which networking hardware is in use, UPnP might also be an option but I have pretty much zero experience with that.