For a Masters course (Mechanical Engineering) I have chosen to design a wireless sensor node. The broad project statement is "Designing a wireless sensor node for big data collection". I'd like to apply this sensor node for collecting information from cnc bending machines at the company I freelance for. The goal of this course is getting some practical hands on experience with microcontrollers.
The metrics that are of interest for this sensor node are:
Machine productivity, measured in products/hour ranging from 150/hour up to 4000/hour
Setup time, measured in minutes or hours
Machine utilization, this is composed of multiple variables and results in expected operational hours/year
The following sensors where chosen after analyzing the machines and identifying ways to extract the desired metrics, a 14bit 3 axis accelerometer (MMA8451) and a sound detector module (electret microphone amplifier). These were chosen because those sensors could be utilized also in a number of different use cases and could possibly reveal extra information such as a measure of how hard the machines are being pushed. These sensors will either be connected to a atsamd21+atwinc1500 low power wifi module or an nRF52 BLE module. The specific boards have a build in lipo charger.
I could use some advice regarding the transfer of sensor data to a database for "cloud processing" of the raw sensor data to provide realtime KPI information about the machine and still allow for analysis of the raw data to possibly extract more information. As a mechanical engineering student I'm not familiar with the do's and dont's of choosing a database and the limits of sending data of wifi or BLE.
The accelerometer data will be three integers ranging from -8191 to 8191 and until I can test the update rate of the actual sensor I will assume it will be 200Hz (same frequency as my phones accelerometer), I will have to test whether it is actually useful to sample at this rate. The sound detector module has an envelope output which outputs an analog voltage which represents the trace of the sound amplitude, this could be sampled at any desired rate and the minimum required sampling rate is also unknown. For arguments sake lets assume I'd like to sample both sensors at 200Hz and also record the timestamp from an RTC and both store the data on an sd card and send the data to a gateway for further processing/storing. How should I try to setup this data stream? What are my limitations or how should I investigate the limits when I receive the sensors?
If I need to provide more input just let me know, I'm not asking for you to complete my project but having some advice would really help me out so I don't spend to much time learning stuff like MySQL and PHP only to discover that I don't have the resources to accomplish my goals using arduino sensor node examples that have data rates of >1Hz...
As you want WiFi do have a look at ESP8266 based boards.
Much more powerful processor than the Arduino, built-in WiFi, but less I/O ports (based on what you listed this doesn't sound like an issue).
There exists a MySQL connector allowing you to send data directly to the database. In my case it takes a few seconds for a single connection to post data to the database, very long. Not sure what's wrong there, it should be faster but 200 Hz is impossible.
If you want really that kind of data rates your best option may be to store it all onto a local SD card, then at the end of the day have the processor transmit the whole log in one go.
As you want WiFi do have a look at ESP8266 based boards.
Much more powerful processor than the Arduino, built-in WiFi, but less I/O ports (based on what you listed this doesn't sound like an issue).
I also have multiple wemos D1 mini's with the lipo charger shield, so if more processor power is the solution at the expense of battery life then I can move to an ESP8266 or even ESP32 (very new so might run into library issues?). The atsamd21 runs at 48MHz and the nRF52 at 64MHz, so that is much better then the 16MHz of most arduino's (still lower then the 80MHz for ESP8266 and the whopping 240MHz dual core ESP32).
There exists a MySQL connector allowing you to send data directly to the database. In my case it takes a few seconds for a single connection to post data to the database, very long. Not sure what's wrong there, it should be faster but 200 Hz is impossible.
I'm not a 100% sure but have read bad things about that connector, most notably in the safety department. Would a valid way to test the speed of my db and my php script be to send data, incrementing the variable by 1 every iteration, and then roughly setting the frequency by using delay() and then seeing how many data is added to the database in a 10s window? So starting with roughly 1Hz the db should have 10 entries and if it could keep up it should have 2000 entries at 200Hz. This should tell me the limit, not what is actually causing this limit.
If you want really that kind of data rates your best option may be to store it all onto a local SD card, then at the end of the day have the processor transmit the whole log in one go.
I have considered this option as I can imagine it would be ideal for power consumption. However if time permits I would like to make a web portal where the current machine status can be displayed. This means there would be 12-20 nodes streaming data to a gateway which processes the data to display the desired KPI with a refresh rate of 0.5-1Hz. This will not be possible if the data is not streamed live...
For the current state - assuming it's not changing too fast, and it's meant for humans to process - you could consider to send it every 10 minutes or so, or maybe every half hour even, with allowance for alarms if something special happens.
wvmarle:
For the current state - assuming it's not changing too fast, and it's meant for humans to process - you could consider to send it every 10 minutes or so, or maybe every half hour even, with allowance for alarms if something special happens.
This could work if I move the data processing to the arduino. The idea is that the vibrations created by the machine have a certain repetition and from this the cycle time can be found and maybe even more information. A challenge is the changing nature of the vibrations, so different products might require different thresholds which are unknowns. This is best handled by more dedicated hardware running a python or matlab script to analyze the data.
I played around with the test data set I recorded with my phone and I could fairly easily detect every cycle if I went down to 50Hz (sampled my data set to emulate the lower sampling rate). Going any lower meant that the chances of missing a cycle start/end became a problem. Although I might not have been using the best signal analysis method.
So I am still left with the problem of having to record and send data within a 20ms window.
I think you should indeed seriously consider moving the data processing to the local nodes, considering the amount of data you want to process. Arduinos are probably not suitable for this work, but more powerful hardware like the Raspberry Pi can definitely do this.
Transmitting all that data wirelessly is an issue - you'd probably have to open a channel and treat it as a constant stream on the receiving side, something like that.