Is it possible to store multiple sensor data on sd card?

Using Arduino Storage
1

Hi,

I'm working on a project and I have various NRF24L01 transmitters (Arduino Nano) sending their data to one single receiver interfaced with a RTC (Real Time Clock) (Arduino Mega).
I need to store the data+the time of receipt on separate files so that I could get a cumulative of the data per hour and to be displayed. The main challenge is the data comes in from all nodes at the same time!!

So, my question is: "Is it possible for one to write into multiple files at the same time every second??"

Find below the Receiver code for reference and edition!

#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>
#include "RF24.h"
#include "printf.h"

RF24 radio(53, 48); // CE, CSN

#define PloadSize  32  // 32 unsigned chars TX payload
byte pipe;
byte CurrentPloadWidth;
byte newdata;
unsigned char rx_buf[PloadSize] = {0};

struct Sensor1Data {
  float v1;
  float c1;
  float p1;
  float k1;
} Sensor1;

struct Sensor2Data {
  float v2;
  float c2;
  float p2;
  float k2;
} Sensor2;

struct Sensor3Data {
  float v3;
  float c3;
  float p3;
  float k3;
} Sensor3;

unsigned char ADDRESS2[1] = {0xb2};
unsigned char ADDRESS3[1] = {0xb3};
unsigned char ADDRESS4[1] = {0xb4};
unsigned char ADDRESS5[1] = {0xb5};


unsigned char ADDRESS1[5] = {0xb1, 0x43, 0x88, 0x99, 0x45}; // Define a static TX address

unsigned char ADDRESS0[5] = {0xb0, 0x43, 0x88, 0x99, 0x45}; // Define a static TX address

void setup()
{
  radio.begin();
  printf_begin();
  Serial.begin(115200);
  radio.setDataRate(RF24_250KBPS);
  radio.enableDynamicPayloads();
  radio.setChannel(124);
  radio.openWritingPipe(ADDRESS0);
  radio.openReadingPipe(0, ADDRESS0);
  radio.openReadingPipe(1, ADDRESS1);
  radio.openReadingPipe(2, ADDRESS2);
  radio.openReadingPipe(3, ADDRESS3);
  radio.openReadingPipe(4, ADDRESS4);
  radio.openReadingPipe(5, ADDRESS5);

  radio.startListening();
  delay(10);
}

void loop()
{
  if (radio.available(&pipe)){
    
    // Fetch the payload, and see if this was the last one.
    CurrentPloadWidth = radio.getDynamicPayloadSize();

    // If a corrupt dynamic payload is received, it will be flushed
    if (!CurrentPloadWidth) {
    }
    else
    {
      radio.read(rx_buf, CurrentPloadWidth);
      newdata = 1;
    }
  }

  if (newdata == 1) {
    newdata = 0;
    if (pipe == 1 && CurrentPloadWidth == sizeof(Sensor1))
    {
      memcpy(&Sensor1, rx_buf, sizeof(Sensor1));

      Serial.println("Data from Sensor 1: ");
      Serial.println(Sensor1.v1);
      Serial.println(Sensor1.c1);
      Serial.println(Sensor1.p1);
      Serial.println(Sensor1.k1);
    }

    if (pipe == 2 && CurrentPloadWidth == sizeof(Sensor2))
      {
      memcpy(&Sensor2, rx_buf, sizeof(Sensor2));

      Serial.println("Data from Sensor 2: ");
      Serial.println(Sensor2.v2);
      Serial.println(Sensor2.c2);
      Serial.println(Sensor2.p2);
      Serial.println(Sensor2.k2);
      }

      if (pipe == 3 && CurrentPloadWidth == sizeof(Sensor3))
      {
      memcpy(&Sensor3, rx_buf, sizeof(Sensor3));

      Serial.println("Data from Sensor 3");
      Serial.println(Sensor3.v3);
      Serial.println(Sensor3.c3);
      Serial.println(Sensor3.p3);
      Serial.println(Sensor3.k3);
      }

    Serial.println("");
  }
  delay(100);
}

Thanks...

2

Dondoter:
I need to store the data+the time of receipt on separate files so that I could get a cumulative of the data per hour and to be displayed.

Why dont you want to write all the data in one file? If I would like to get the cumulative value of one hour I would not read the data from the SD card but do the calculation every time I receive data.

3

I actually can't consider saving all the data into one file because of the way I want to display the data!

4

Dondoter:
So, my question is: "Is it possible for one to write into multiple files at the same time every second??"

Try it. I've a feeling that the library only permits four files open at a time, but I can't find anything in the documentation that confirms it.

If you can't use six, use one as suggested and write something to post process your file into whatever form you need for your display. I'd do that step on whatever computer you're using to program the mega, but you could equally well make the mega do it.

5

Dondoter:
I actually can't consider saving all the data into one file because of the way I want to display the data!

Probably nonsense. But you say nothing about how you actually want to display the data.

The main challenge is the data comes in from all nodes at the same time!!

Also nonsense, the data appears to be received sequentially, the intervals being uncertain, as you have a 100ms delay in the loop but talk about one second. There is no point in guessing about what is really going on, or what you really want, but the most likely time waster you are going to encounter is the opening and closing of sundry SD files, which makes this a rather good time for thinking more sensibly about file management.

6

Nick_Pyner:
Probably nonsense. But you say nothing about how you actually want to display the data. Also nonsense, the data appears to be received sequentially, the intervals being uncertain, as you have a 100ms delay in the loop but talk about one second. There is no point in guessing about what is really going on, or what you really want, but the most likely time waster you are going to encounter is the opening and closing of sundry SD files, which makes this a rather good time for thinking more sensibly about file management.

I get your point!

  1. For display, I'm going to be reading the files from the SD card into a database developed using the influxDb platform from which a Grafana Dashboard will display the readings!
  2. Due to the fact that I would require each node to give its own readings and total over time, i thought it best to have separate files so as to avoid misrepresentation of data

Hope that makes sense!!
thanks...

7

Not much sense, I'm afraid...

For display, I'm going to be reading the files from the SD card into a database developed using the influxDb platform from which a Grafana Dashboard will display the readings!.

I have never heard of influxDb or graphana but, if it makes such demands on the way you input data, that is a good reason why I haven't.

i thought it best to have separate files so as to avoid misrepresentation of data

It probably isn't. You should able to sort this stuff out at the destination, the most obvious means being via Excel. The fact that you mention SD at all means this is not some real-time plotting venture, and you have the time to do this.

8

LowArt:
Why dont you want to write all the data in one file? If I would like to get the cumulative value of one hour I would not read the data from the SD card but do the calculation every time I receive data.

The thing is; the data transmitted is coming from different nodes and it at display I need to be able to identify the data from each one!

9

Dondoter:
I need to be able to identify the data from each one!

This is surely just a matter of properly naming the incoming data, which is probably needed for the display you vaguely mention anyway. How this can possibly be complicated, or need separate files? The same matter applies if all the data was coming from a single source.
You may come up with a good reason for doing what you propose, but I doubt you ever will, and I submit you are just trying to make things more complicated than they need to be.

10

Nick_Pyner:
This is surely just a matter of properly naming the incoming data, and probably need for the display you vaguely mention anyway. How this can possibly be complicated, or need separate files? The same matter applies if all the data was coming from a single source.
You may come up with a good reason for doing what you propose, but I doubt you ever will, and I submit you are just trying to make things more complicated than they need to be.

That's a good point!
Please could you possibly share a better or at least more efficient yet simpler idea on how to overcome the challenge of displaying real-time data transmitted?

I genuinely don't have any better thing in mind, hence why I threw the question here!!
thanks...

11

I can't comment much further as your code looks quite alien to me. But, as I understand it, the objective is to acquire values, assign them to variables, and record same on an SD - just like lots of other people do, including me. If you use a name for the variables that includes relevant information, like the source, you shouldn't have a problem. If you are sending the data to a spreadsheet, the variable name is trivial, what counts is the order the values are sent, and the naming is done at the other end.