I have a bunch of ESP8266 board scattered around the house, sometimes they stop working and it takes a lot of time to figure out what’s the problem because I have no idea what is going on. I already use the serial console to print out some useful information but they are unusable when the board is deployed in its final position.
I thought about hosting a web page on each board and post the messages on there but if I can’t reach the device and I need to restart it I will loose the log.
Then I thought about hosting a database, it would be relatively easy to host because I already have a server but it would require a lot of time to learn how it works and implement so I wanted to know if this was a good idea or if there was a better way to do it
let each esp send a heartbeat message to the server, e.g. each 5 minutes, or if you want more live activity in a higher frequence.
let the server check if all esp are alive, if not inform you about the missing device
I use plain http / post messages between ESP and Server for this.
The ESP don't care if there is a database on the server or if the server stores the data in flat files. Use what you have available and what comes easier for you.
Send whatever log messages your ESPs generate to a central server and store them. Initially (and perhaps forever), text files will be fine. Maybe eventually you'll want to learn to use MySQL or the like.
You can have a web site on the server that collects the messages and stores them or some more custom code that accepts data over UDP.
As ever, start with whatever you find simplest to do and enhance it if you have to.
I think that a database would be neater and maybe a little bit simpler to implement than a custom solution but I haven’t made a lot of research so I might be wrong. The important thing to me is to know whether or not there is an off the shelf solution before I spend a lot of time learning something new.
It might help if you were just a little bit more forthcoming about the purpose of these ESP8266s that are scattered about the house. By virtue of being an ESP8266, one might imagine they communicate data by WiFi to a point and, in that event, all you have to do is note and record that they fail to do so. How hard can that be? This should require no more than a properly employed SD card - something that you may already have in hand. But who would know?
If it is also serving as an AP would you still not be able to reach it ? That would mean it would be stuck in an endless loop somewhere, or your WiFi connection is not working at all anymore, possibly due to interrupts being turned off at some point in your program.
Using MQTT connections, one would assume that the error lies there, (it tends to be the case with MQTT that reliability is not superb) but that shouldn't have to influence any webserver running on the same unit.
Should be easy enough to sort out once you setup OTA updates on all units. Then they can remain in place while developing.
I don't think there's a standard solution here - there must be fifty ways to leave your lover do your logging.
Since you are already using MQTT, you might consider having the logging server subscribe to the topics your ESPs are writing to and echo that to a file. Maybe add a logging topic too.
I already send some messages via MQTT but I would like to avoid sending too many messages because the MQTT console it is already quite messy.
I would like a separate log server where I can send as much messages as I want so hopefully I can debug and reupload the code remotely around the house
There's always the "noDB" alternatives such as Splunk. They have a free version that allows you to index 500 MB worth of data per day. If you know how to send a UDP packet from an ESP, then Splunk is as easy as activating a UDP receiver port. No need to build databases and tables, and schemas etc.
It's as easy as...
ESP.createUDPMessage("This is a test message");
ESP.sendUDP(message, 514); << I used port 514 since that's the standard syslog port