My problem is the following, I am developing a maintenance system based on vibration analysis, to do the analysis I perform the FFT of the signal read by the vibration sensor (Accelerometer ADXL345). With my current accelerometer I can read 1600 values in one second. Every 30 seconds my arduino starts reading the accelerometer and as soon as it finishes reading the 1024 values it needs to send the data to a Gateway which will send the data to a web server.
Okay, that's the idea, but how can I do this in practice?
How can I gather 1600 values and send them all at once using an NRF24L01 for the Gateway?
Is there any way to optimize this whole system?
I thought about putting everything together in a Json and sending it, but I'm not succeeding because ArduinoJson doesn't allow you to store so much data in an Array, and when I use DynamicJson it returns empty.
THE FFT WILL BE DONE ON THE SERVER WHEN THE DATA ARRIVES
#include <ArduinoJson.h>
#include <ArduinoJson.hpp>
#include <SPI.h>
#include "RF24.h"
#include "EmonLib.h"
#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_ADXL345_U.h>
#include "Adafruit_MCP9808.h"
#include <SoftwareSerial.h>
#include "LowPower.h"
#include "MAX17048.h"
#define DEBUG_LED 9
#define LED_ERROR_RADIO 3
#define LED_ERROR_SENSOR 2
#define LED_SUCCESS 1
#define LED_FAILURE 0
#define DELAY_FOR_DATA_TRANSFER_SEG 30
#define SENSOR_KEY "0000A"
RF24 radio(7, 8);
Adafruit_ADXL345_Unified accel = Adafruit_ADXL345_Unified();
Adafruit_MCP9808 tempsensor = Adafruit_MCP9808();
MAX17048 pwr_mgmt;
const byte address[6] = "00001";
#define SAMPLES 1024 //Must be a power of 2
#define SAMPLING_FREQUENCY 1600 //Hz. Determines maximum frequency that can be analysed by the FFT.
unsigned int sampling_period_us;
unsigned long microseconds;
int delay_int = 0;
void aDXL345Setup() {
if (!accel.begin()) {
Serial.println("[ERROR] No ADXL345 detected...");
//blinkLed(LED_ERROR_SENSOR);
while (1)
;
}
Serial.println("Found ADXL345!");
accel.setRange(ADXL345_RANGE_16_G);
accel.setDataRate(ADXL345_DATARATE_3200_HZ);
}
JsonArray readVibration(DynamicJsonDocument doc){
JsonArray vibration = doc.createNestedArray("vibration");
sensors_event_t event;
accel.getEvent(&event);
vibration.add(event.acceleration.x);
vibration.add(event.acceleration.y);
vibration.add(event.acceleration.z);
return vibration;
}
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
aDXL345Setup();
sampling_period_us = round(1000000 * (1.0 / SAMPLING_FREQUENCY));
delay_int = round(DELAY_FOR_DATA_TRANSFER_SEG / 8);
}
int idleTimer = 800;
void loop() {
// put your main code here, to run repeatedly:
idleTimer += 1;
//if (idleTimer >= delay_int) {
//radio.powerUp();
//sendData();
sendData();
//radio.powerDown();
//idleTimer = 0;
//}
delay(30000);
//LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);
}
void sendData() {
StaticJsonDocument<1024> doc;
doc["key"] = "0000B";
doc["battery"] = 48.75608;
doc["temperature"] = 48.75608;
JsonArray fft = doc.createNestedArray("fft");
for (int i = 0; i < SAMPLES; i++) {
microseconds = micros();
sensors_event_t event;
accel.getEvent(&event);
fft.add(sqrt(event.acceleration.x * event.acceleration.x + event.acceleration.y * event.acceleration.y + event.acceleration.z * event.acceleration.z));
//Serial.print("Accel Y: "); Serial.println(sqrt(event.acceleration.x * event.acceleration.x + event.acceleration.y * event.acceleration.y + event.acceleration.z * event.acceleration.z));
while (micros() < (microseconds + sampling_period_us)) {}
}
String json;
serializeJsonPretty(doc, json);
Serial.println(json);
}
Why are you using this gateway and what model of gateway is it? Can you not simply use a WiFi enabled Arduino and send the data directly?
What format is the gateway expecting the data to be in, and what format is the server expecting the data to be in?
JSON is an inefficient format for sending data in bulk, because all the values are transmitted as ASCII text, plus there is the overhead of field names and delimiters.
You have made the classic beginner error of forgetting to mention what type of Arduino you are using. If it is something basic like Uno or Nano V3, then it may not have enough memory to store the 1024 values in binary format. It certainly won't have enough memory to store the data in JSON format.
In the industrial environment where the sensors are installed, there is no WiFi, so the Gateway is used to gather data from the various sensors and send it via 4G to the Server (data from each sensor is separated and stored according to its Sensor_Key). I'm not using the arduino board itself, but an Atmega328p-au chip in the sensor and Atmega328p in the Gateway.
The data format of the server and gateway will depend on the solution I have for the sensor, I can change them to the most efficient way. The only thing that is fixed is how the data is sent, which is via an NRF24L01.
Atmega328 has only 2KB of RAM memory, and you can't use all of that because some of it is needed for other things like buffers. Even if your 1024 values need only 2 bytes each in binary format (and I suspect they need more than that) there is not enough memory.
In JSON format, each value is probably going to need 10~15 bytes to store it, so I suspect your message is going to be at least 15KB.
You could try to transmit the data as it is collected from the sensor, but I strongly suspect you won't be able to hit that 1600 values per second speed if you do that. You would also be unable to use that ArduinoJSON library, because it is designed to construct the whole message in memory before it is transmitted.
So the only ideas I have right now are to find a way to expand the memory of the armega328s (not easy) or replace them with something with significantly more available memory.
Yes, I had thought about the memory impairment, but I was still trying to get at least 128 pieces of data, the right amount would really be 1024.
I had an idea, please tell me if it's possible:
Store the data in an external EPROM as it is read, the only function of the atmega would be to read it and send it to the epron, without having to store anything in an array.
And then read the EPROM and send it to the gateway (in the case of the gateway, you'd really have to change the processor since it will need to store the data in order to send it to the server).
In the project, there is already an EPROM installed in the pcb that was used to store data that was lost when it was sent. With a few code adaptations, it would already be possible to use it (24LC256-I/MS 256 kbit).
EEPROM has limited write-cycles (eg. 10,000) and is quite slow to write, especially if you use i2c.
FRAM has far more write-cycles, but I suspect it's also slow.
Ideally you would use external SRAM with an SPI interface for speed.
But even with that, you still have a problem with the ArduinoJSON library. It needs enough internal RAM to write the whole document. You may have to abandon the library and write the JSON data "on the fly" one line at a time, transmitting each line before writing the next. I suspect that's not that difficult, actually.
Then your gateway may have a similar problem. It will need to receive the JSON data one line at a time and send it to the server one line at a time, because it won't have space to store the whole document. Again, that sounds achievable.
The ATmega328P is a poor choice for this application, because of the extremely limited RAM memory and slow processor speed. There are plenty of other options, many with built in WiFi.
Okay, in my current scenario, it would be possible to record 128 pieces of data, for example, and send them one by one to the gateway, right?
I don't have a defined format yet, json was just an example. Do you think bytes would be more efficient? Could it be sent all at once?
I didn't put the general situation into context. In this case, the sensor was used to record a single vibration and temperature value every 30 seconds, which worked very well. Until they wanted to implement the FFT system, now I need to take several data at once.
And using WiFi isn't viable. 4G works fine, but you need a gateway.
In the future, when I use it with larger machines of up to 12kHz, I will probably need to modify the current PCB I have. Do you have a processor in mind that could be used?
Yes, that should be achievable. But not by using the ArduinoJSON library.
Whatever method you use, it's always going to be bytes! The real question is, how is the data encoded in those bytes?
Encoding the data in binary is always more efficient. Like I said before, each value is probably going to be either 2 or 4 bytes in binary. But in JSON/ASCII, it's probably going to be 10~15 bytes. Like that, you won't have room for even 128 samples.
Another classic beginner error. That's what you should always do in your first post.
The Arduino compatible Teensy 3.2 has enough memory and a good, 12 bit ADC that is fast enough to digitize at rates over 200 kHz. For accurate spectra, most people digitize data at about 10x the highest frequency of interest.
looking back at your original specification it is similar to a commercial project from a couple of years ago
processor PIC24FJ256GA705 using two 1000byte arrays concurrently running
every 20mSec interrupt: reading a LSM9DS1_magnetometer adding 19byte sample to an array (18bytes data + sample counter)
every second arrays were switched and 1000byte frame transmitted over BLE to an Android phone for analysis
each frame consisted of frame header + sequence number + data + CRC + frame end (data and CRC was byte stuffed)
when processed would look like
Acceleration X: -0.67 m/s^2 Y: 1.23 m/s^2 Z: 9.64 m/s^2
magnetic X: -0.15 m/s^2 Y: -0.05 m/s^2 Z: -0.07 m/s^2
Gyroscope X: 0.01 m/s^2 Y: 1.58 m/s^2 Z: -1.99 m/s^2
think I would look at using a more powerful microprocessor than a UNO
e..g. ESP32 and use WiFi or bluetooth to transmit data for analysis
what processor is doing the FFT?
