I am making a system to monitor several temperature and humidity sensors and save the data with timestamps. Ideally, I want to be able to walk up to the system and get the data in either a table or graph format on my phone or tablet. I have to think of a way to make it work with or without an internet connection. I'm bouncing ideas off the group to get some ideas.
For data collection, I poll the sensors every 10 minutes and save the readings and timestamp to a text file on a SD card.
For the no Internet situation, I add a Bluetooth circuit and let the user pair their device to the Arduino. It would require a special app, but if they press a button in the app, it talks to the Arduino, which then retrieves the data from the SD card and sends it to the phone or tablet. The app then presents it in columns or as a chart.
For the Internet enabled system, I'm thinking that it could simply send the data to the cloud and it would then be viewable on a webpage.
I'm sure all this has been done in multiple ways, but I'm seeking input from members who may have done this already.
ScottMurchison:
I am making a system to monitor several temperature and humidity sensors and save the data with timestamps.
I'm sure all this has been done in multiple ways, but I'm seeking input from members who may have done this already.
You're dead right about that, it is one of the things Arduino does best. Check out Bluetooth Graphics Terminal, it's God's own phone app. Also the dumpfile example in the IDE. Then Adafruit i/o for Internet of Things.
I would start by making a list of what you want to measure and what sensor you are going to use.
Consider using Unix time for your timestamp with a real time clock, there are many sketches that show how to do that.
Determine how to update the RTC when you connect to ???.
Determine how to format the data you want to save.
Determine how to format the data you want to send.
Determine how much NVM (Non Volatile Memory) you need.
Do the same for your target application.
Write code for each piece of hardware and debug it. Save this code for later testing if needed.
Now format your transmit string and verify it is being formed correctly.
Now stare on one of the apps, and follow basically the same procedure.
Document each step, saves lots of time when there are problems.
This is not inclusive, just a starting point.
I have had a system very similar to what you describe operating continuously for a couple of years, collecting weather data. There are three remote Arduinos, each collecting data from an array of sensors local to it. The master controller is a Raspberry Pi, which receives a data package from each remote Arduino every minute. Communication is via serial link, also, one of the remotes is linked via nRF24L01 wireless transmitter. The Pi also controls a 16 bit ADC which gathers more data from analog sensors proximal to it. The Pi coordinates all of the data, assigns a timestamp, and stores the data to a USB drive.
The master controller is a Pi because it has considerably more built-in capabilities in terms of timestamping, data display (either tabular or graphical, using matplotlib), remote logins for ftp and ssh sessions, a ready-made file operating system, and more or less unlimited program space. The choice was a no-brainer. For the remotes, on the other hand, Arduinos were the natural choice, as they are relatively small and power frugal, and none of the attributes listed above are necessary in those roles.
This system has grown considerably in complexity over time, but I have never had reason to doubt the fitness to task of the basic architecture.
I have a lot to work out. The application is a hops dryer, with a heater, fan, display and a few other options. The plan is to have five DHT22 temp/humidity sensors to read data from. I am prototyping the system using an Arduino Mega with a TFT LCD screen to display status and readings. I have added a DS3231 RTC and may add the Bluetooth and SD card board next.
I'm looking at all the replies above to get the best ideas incorporated. One issue is that the dryer will be out in a barn and I have to assume that it may be out of reach of WiFi. That doesn't mean that I couldn't use LoRa or something similar to beam the data to another Arduino in the house, but that may be too complex for the average user to use.
Saving data on a regular USB stick is another option, but it requires another board and mounting the circuitry in such a way that it can be accessed easily on the front panel by the user.
ScottMurchison:
The plan is to have five DHT22 temp/humidity sensors to read data from.
I don't know anything about DHTs, but I understand they are MickeyMouse devices for both temperature and humidity. If you are serious about this, you might investigate DS18B20 as wet/dry bulb devices. You should at least closely investigate the DHTs to see if they are suitable.
One issue is that the dryer will be out in a barn and I have to assume that it may be out of reach of WiFi. That doesn't mean that I couldn't use LoRa or something similar to beam the data to another Arduino in the house, but that may be too complex for the average user to use.
It can't be hard to check the viability of WiFi before you go down the LoRa road. WiFi and Bluetooth are so much better for talking to the outside world. Some Bluetooth 4 devices are good for 100m. You might be even better off using a phone board than LoRa.
Saving data on a regular USB stick is another option, but it requires another board and mounting the circuitry in such a way that it can be accessed easily on the front panel by the user.
Indeed, you will require a USB host board. If you are going to stand by the Mega to download, you might as well do it via bluetooth. Note that you might be better off with Bluetooth on flying leads for this.
Nick_Pyner:
I don't know anything about DHTs, but I understand they are MickeyMouse devices for both temperature and humidity.
My experience with the DHT22 temperature output has been reasonably good, but I'm much less confident about the humidity readings. The ones I have do not survive a high humidity outdoor environment. I'd definitely suggest adding at least one other type sensor, eg BME280, for comparison. Unfortunately the other sensors can be significantly more expensive, but maybe you get what you pay for. It's not that much money considering the time you put into the project.
S.
Node-RED uses Browser-based flow editing with a vast range of nodes for file access, databases, email, network, buttons, graphs, gauges ... You can write your own functions in Javascript. There are many tutorials and Youtube videos available for this.
I would recommend an Arduino with WiFi and Bluetooth e.g. Arduino Nano 33 IoT or and ESP32 board depending what else you need to do.
When you have WiFi you can send data using MQTT in real time, but you will need power because batteries only last for a short amount of time. MQTT is a lightweight protocol on top of TCP/IP and is supported from Arduino, to Raspberry Pi and up to the cloud. It requires a broker node e.g. a Raspberry Pi or a cloud service.
If you want to save power and store data locally I would recommend SD cards over USB sticks because you would need USB Host support.
With a modern microcontroller you can also keep more data in RAM. The Nano 33 IoT has 32KB SRAM over the 8kB of your Mega. You might not need an SD card if you do not mind losing the data when the power is out.
You could look at my paid pfodApp which both plots and logs data.
There is a free pfodDesigner app that will generate the Arduino code for you.
The pfodApp is a general user interface that is completely controlled by the code in your Arduino so very flexible.
pfodApp will connect via WiFi, Classic Bluetooth, BLE and SMS
Lots of example projects on the pfodApp website. Including a high speed SD data logging project.
The DHT22's humidity detection tends to get stuck on 100% in high humidity environments. A single night where it reaches 100% may be enough for that to happen.
with an ESP and a cell battery pack, acting as an AP on it's own,
you can connect your phone to the ESP as a wifi device
there are example sketched that will tell you what wifi device it picks up and also the strenght.
the ESP-32-S can tell you the distance to the Wifi hub or device.
and, LoRa is not something only for those with deep experiance, but you can do that if you need.
As for the humidity, I use the BME280 (not bmP280) the p only does pressure, the E does humidity too.
interesting idea to use a pair of Ds18B20's with a sock for wet bulb dry blub. been awhile since I slung a psychrometer around.
wvmarle:
The DHT22's humidity detection tends to get stuck on 100% in high humidity environments. A single night where it reaches 100% may be enough for that to happen.
That is precisely my experience as well. I'm hoping for something better from the Bosch line of sensors (e.g. BME280), as suggested by Idahowalker. I do have a DHT22 indoors which appears to still be functioning normally after 1+ years, in an environment which rarely strays outside of the 35-60%RH range. That's not a very demanding application.
S.
