Go Down

Topic: data classification with arduino (Read 429 times) previous topic - next topic

Farnaz_201

I want to classify my data from 3 sensors with arduino so that every time i enter the specific date or hour, or also it can be a period of time like 10 days ago, it shows me the exact data. In other word i want to classify my data so that whenever i want every detail of data, i would be able to get it by entering the information of year,date,time. I also didn't want my data to be in separate files and time stamps. I want to be able to do that within one file and search inside that file...

UKHeliBob

What have you tried so far ?

Have you got code to save the data to a file, presumably on an SD card ?  Does the data include a timestamp and if so what format is it in ?
Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

Robin2

I also didn't want my data to be in separate files and time stamps. I want to be able to do that within one file and search inside that file...
If you want to classify data by date and time then you need timestamps. If you don't want a separate file for  each hour (or day or whatever) then the time and date must be saved alongside each item of data within the single file.

An Arduino is great for collecting data but it's not really ideal for viewing data afterwards - it does not have a much memory,  it does not have a convenient display and C++ is not the easiest language for writing code to view data. I reckon it would make more sense to copy the data to your PC and do the analysis on the PC, perhaps using Python.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

Farnaz_201

What have you tried so far ?

Have you got code to save the data to a file, presumably on an SD card ?  Does the data include a timestamp and if so what format is it in ?
I have saved my data from 3 different sensors into a file in datalogger shield. I want to send a request for data received (probably containing start and end time stamps, parse through the file, and start returning information for those date/time stamps within the requested start/end time stamps.

Farnaz_201

If you want to classify data by date and time then you need timestamps. If you don't want a separate file for  each hour (or day or whatever) then the time and date must be saved alongside each item of data within the single file.

An Arduino is great for collecting data but it's not really ideal for viewing data afterwards - it does not have a much memory,  it does not have a convenient display and C++ is not the easiest language for writing code to view data. I reckon it would make more sense to copy the data to your PC and do the analysis on the PC, perhaps using Python.

...R
I saved the file in SD card datalogger. So i guess the memory won't be a problem. The things i don't know is how to use timestamp and how to call the timestamp i need at that time...

sterretje

Do you store the timestamp (in whichever format) with the data? A timestamp (for what you want) will come from a RTC or a NTP server.
If you understand an example, use it.
If you don't understand an example, don't use it.

Electronics engineer by trade, software engineer by profession. Trying to get back into electronics after 15 years absence.

Farnaz_201

Code: [Select]
#include "RTClib.h"
#include <SD.h>
#include <SPI.h>

RTC_DS1307 rtc;
const int chipSelect = 10;

File dataFile;

String print_time(DateTime timestamp) {
  char message[120];

  int Year = timestamp.year();
  int Month = timestamp.month();
  int Day = timestamp.day();
  int Hour = timestamp.hour();
  int Minute = timestamp.minute();
  int Second= timestamp.second();

  sprintf(message, "%d-%d-%d %02d:%02d:%02d", Month,Day,Year,Hour,Minute,Second);
 
  return message;
}

void setup(){
  Serial.begin(9600);
 
  pinMode(chipSelect, OUTPUT);
  if (!SD.begin(chipSelect)){
    Serial.println("Error: SD card would not initiate.");
  }
 
  rtc.begin();
  if (!rtc.isrunning()){
    Serial.println("Clock is not running");
  }
 
  dataFile = SD.open("log0.csv", FILE_WRITE);
  if (!dataFile){
    Serial.println("Could not open file.");
  }
 
}

void loop(){
  Datetime now = rtc.now();
  dataFile.println(print_time(now));
 
  delay(3000);
}

Farnaz_201

Do you store the timestamp (in whichever format) with the data? A timestamp (for what you want) will come from a RTC or a NTP server.
I wrote a code with timestamp and data. Now,how can i ask the arduino to send me the timestamp i want? I uploaded my code. Now i ask the arduino to send me the last 10 timestamps.... any suggestions?

UKHeliBob

Quote
I wrote a code with timestamp and data
Have you posted it here ?
Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

sterretje

Does your code in reply #6 actually write something useful to the card.

The print_time function is supposed to return a String but actually returns a pointer to a character array. I have my doubts that that will work correctly.
If you understand an example, use it.
If you don't understand an example, don't use it.

Electronics engineer by trade, software engineer by profession. Trying to get back into electronics after 15 years absence.

Robin2

#10
Dec 10, 2019, 04:43 pm Last Edit: Dec 11, 2019, 12:14 pm by Robin2
I saved the file in SD card datalogger. So i guess the memory won't be a problem.
There will be plenty of space to store the data on the SD Card but your analysis of the data still has to work within the limited Arduino SRAM.

Quote
The things i don't know is how to use timestamp and how to call the timestamp i need at that time...
A good starting point would be to post a two or three sample messages in the format that they exist on the SD Card. That way we can see exactly what you are trying to work with.

...R

PS ... what about just moving the SD Card to a PC and taking a copy of the file?
Two or three hours spent thinking and reading documentation solves most programming problems.

Farnaz_201

Does your code in reply #6 actually write something useful to the card.

The print_time function is supposed to return a String but actually returns a pointer to a character array. I have my doubts that that will work correctly.
It just shows me the time. I can add other data later, but the question is can i type a particular date and time in serial monitor and ask it to show me the data with the timestamps?

Farnaz_201

There will be plenty of space to store the data on the SD Card but your analysis of the data still has to work within the limited Arduino SRAM.
A good starting point would be to post a two or three sample messages in the format that they exist on the SD Card. That way we can see exactly what you are trying to work with.

...R

PS ... what about just moving the SD Card to a PC and taking a copy of the file?
It's not actually the part of the project to remove the card... i should be able to transfer data with timestamp via bluetooth module. let me explain it again... I have 3 different sensors which their data is stored in my SD card with particular interval time. Now i want to see the last 10 or 20 data from yesterday. I should print the period of time i want into the terminal via bluetooth module and after that i get that data box i want. I wrote the code to get the sensor's data and save it to SD card with their own timestamp(using RTC module on SD logger shield), but i don't know how can i communicate with arduino to let it know the timestamp i want and send it to me... I want to type for example september 23-september25 and then ask for the data of this period in return... I hope i'm clear with the explanation...

Farnaz_201


UKHeliBob

A Bluetooth module such as the HC05 will communicate with the Arduino via a serial interface.  Have you  got a Bluetooth module ?  Which Arduino are you using ?

Please post a sample, say 10 records, of the data that you are currently storing
Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

Go Up