Data Logging Problem - TimeStamp Stop Working After Write Data to SD Card

enormousdreamer · June 6, 2015, 10:47am

I have some problem for my project. I know that Yun Board is not supported by RTC. However, we can use bridge to access the time from the network (using AR933), while the Yun is connected to Internet.

Well, I've found the basic code to access the timestamp from the bridge. Here's the code (I add little code to act as data)

#include <FileIO.h> //Untuk komunikasi dengan SD card

void setup() {
  //Bridge
  Bridge.begin();
  
}

String getTimeStamp() {
  String result;
  Process time;
  // date is a command line utility to get the date and the time 
  // in different formats depending on the additional parameter 
  time.begin("date");
  time.addParameter("+%D-%T");  // parameters: D for the complete date mm/dd/yy
                                //             T for the time hh:mm:ss    
  time.run();  // run the command

  // read the output of the command
  while(time.available()>0) {
    char c = time.read();
    if(c != '\n')
      result += c;
  }

  return result;
}

void loop() {
  String Data;
  Data += getTimeStamp();
  Data += ",";
  Data += "korewa-korewa";
  Serial.println(Data);
  delay(1000);

}

And from that code the serial monitor shows the timestamp and the additional data.

06/06/15-17:34:50,korewa-korewa
06/06/15-17:34:51,korewa-korewa
06/06/15-17:34:52,korewa-korewa
06/06/15-17:34:53,korewa-korewa
06/06/15-17:34:54,korewa-korewa
06/06/15-17:34:56,korewa-korewa
06/06/15-17:34:57,korewa-korewa
06/06/15-17:34:58,korewa-korewa
06/06/15-17:34:59,korewa-korewa
06/06/15-17:35:00,korewa-korewa
06/06/15-17:35:01,korewa-korewa
06/06/15-17:35:02,korewa-korewa

Just like what I want.

Here is the problem. I want to keep those data in the SD Card by using the following code

#include <FileIO.h> 

void setup() {
  //Inisialisasi Bridge
  Bridge.begin();
  
  //Inisialisasi komunikasi dengan file sistem OpenWrt-Yun
  FileSystem.begin();
}

String getTimeStamp() {
  String result;
  Process time;
  // date is a command line utility to get the date and the time 
  // in different formats depending on the additional parameter 
  time.begin("date");
  time.addParameter("+%D-%T");  // parameters: D for the complete date mm/dd/yy
                                //             T for the time hh:mm:ss    
  time.run();  // run the command

  // read the output of the command
  while(time.available()>0) {
    char c = time.read();
    if(c != '\n')
      result += c;
  }

  return result;
}

void loop() {
  delay(1000);
  String Data;
  Data += getTimeStamp();
  Data += ",";
  Data += "korewa-korewa";
  Serial.print(Data);
  Serial.println(" - Before writing data to SD Card");
  File dataFile = FileSystem.open("/mnt/sd/timetest1.txt", FILE_APPEND);

  // if the file is available, write to it:
  if (dataFile) {
    dataFile.println(Data);
    dataFile.close();
    // print to the serial port too:
    Serial.print(Data);
    Serial.println(" - After writing data to SD Card");
  }  
  // if the file isn't open, pop up an error:
  else {
    Serial.println("error opening datalog.txt");
  } 
  delay(1000);
}

However the result in the serial monitor and the .txt file (in sd card) is shown below

06/06/15-17:41:34,korewa-korewa - Before writing data to SD Card
06/06/15-17:41:34,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card
,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card
,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card
,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card
,korewa-korewa - After writing data to SD Card

as you can see for the first data the timestamp is recorded, but after that the timestamps for the next data are not recorded.

Could you guys help me ? What's wrong with my code ?

Note : Actually my project is to record the data from several sensors and data logging it to sd card. Those string data in my code above just a little representation of my sensors reading.

Thanks for the help.

sonnyyu · June 6, 2015, 11:57am

Use either datetime.now() or CURRENT_TIMESTAMP to log TimeStamp.

Datalog to csv file with time stamp

http://forum.arduino.cc/index.php?topic=224609.msg1627106#msg1627106

SQLITE3 with CURRENT_TIMESTAMP and Yun

http://forum.arduino.cc/index.php?topic=309101.msg2144694#msg2144694

Angelo9999 · June 8, 2015, 7:12am

Try to declare

String Data;

as global variable, then in getTimestamp function modify directly Data.
And remember at the end of the loop to clear the Data string with:

Data = "";

enormousdreamer · June 8, 2015, 9:36am

Angelo9999:
Try to declare
String Data;
as global variable, then in getTimestamp function modify directly Data.
And remember at the end of the loop to clear the Data string with:
Data = "";

well, thanks a lot, still the Result id not displaying the next timeStamp

#include <FileIO.h> 

String Data;

void setup() {
  //Inisialisasi Bridge
  Bridge.begin();
  
  //Inisialisasi komunikasi dengan file sistem OpenWrt-Yun
  FileSystem.begin();
}

void getTimeStamp() {
  String result;
  Process time;
  // date is a command line utility to get the date and the time 
  // in different formats depending on the additional parameter 
  time.begin("date");
  time.addParameter("+%D-%T");  // parameters: D for the complete date mm/dd/yy
                                //             T for the time hh:mm:ss    
  time.run();  // run the command

  // read the output of the command
  while(time.available()>0) {
    char c = time.read();
    if(c != '\n')
      result += c;
  }

  Data += result;
}

void loop() {
  getTimeStamp();
  Data += ",";
  Data += "korewa-korewa";
  Serial.print(Data);
  Serial.println(" - Before writing data to SD Card");
  File dataFile = FileSystem.open("/mnt/sd/timetest1.txt", FILE_APPEND);

  // if the file is available, write to it:
  if (dataFile) {
    dataFile.println(Data);
    dataFile.close();
    // print to the serial port too:
    Serial.print(Data);
    Serial.println(" - After writing data to SD Card");
  }  
  // if the file isn't open, pop up an error:
  else {
    Serial.println("error opening datalog.txt");
  } 
  Data += "";
  delay(1000);
}

Could you tell me why my code are not working properly ?

sonnyyu:
Use either datetime.now() or CURRENT_TIMESTAMP to log TimeStamp.

Datalog to csv file with time stamp

Another date/time question - #6 by sonnyyu - Arduino Yún - Arduino Forum

SQLITE3 with CURRENT_TIMESTAMP and Yun

PHP, SQLITE3 and Yun - Arduino Yún - Arduino Forum

thanks for the help. However, I'm still confused how to use the code.

Angelo9999 · June 8, 2015, 10:53am

Try to add

time.close();

at the end of the getTimestamp function.

At the end of the loop it should be

Data = "";

instead of

Data += "";

enormousdreamer · June 8, 2015, 11:26am

Angelo9999:
Try to add
time.close();
at the end of the getTimestamp function.

At the end of the loop it should be
Data = "";
instead of
Data += "";

Still no luck, the result is still the same

06/08/15-18:25:18,korewa-korewa - Before writing data to SD Card
06/08/15-18:25:18,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card
,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card
,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card
,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card
,korewa-korewa - After writing data to SD Card
,korewa-korewa - Before writing data to SD Card

sonnyyu · June 8, 2015, 1:49pm

enormousdreamer:
...
thanks for the help. However, I'm still confused how to use the code.

Datalog to CSV file with time stamp

http://forum.arduino.cc/index.php?topic=224609.msg1627106#msg162710

The timestamp takes place at Linux side only. We offload the ATmega32u4 from timestamp function.

Q: Shortest path between point A and point B?
~~A: Connect straight line between them.~~
A: Move point A on point B

ShapeShifter · June 8, 2015, 2:24pm

sonnyyu:
The timestamp takes place at Linux side only. We offload the ATmega32u4 from timestamp function.

Q: Shortest path between point A and point B?
~~A: Connect straight line between them.~~
A: Move point A on point B

This is a very good suggestion. Here's what the original sketch is doing:

Collect some data
Run a Process object to request the time from Linux
Fetch the time from the Process object
Combine the time stamp and the data
Use a FileSystem object to request that Linux opens a file on the SD card
Use the FileSystem object to request that Linux write a line of text to the SD card
Use the FileSystem object to request that Lunux close the SD card

Here's what the sketch could be:

Collect some data
Run a Process object to request that Linux write the data to a file (the data to be written is a parameter to the Process call)

The second method requires writing a Python script on the Linux side. This script:

Gets the system time
Appends the data (parameter to the script) to the time
Writes the data to a file on the SD card.

The first method does a LOT of communications back and forth between the sketch and Linux. The sketch is doing all of the hard work, and the Linux side is basically loafing, simply responding to the sketch's requests.

The second method does a single transfer between the sketch and Linux (the actual collected data.) Not only is this more efficient, but the sketch is doing what it does best (true real time processing and collecting/outputting data) while the the Linux side is doing what it does best (getting time, data manipulation, file I/O, etc.)

enormousdreamer · June 23, 2015, 7:22am

sonnyyu:
Use str(datetime.now())

touch /mnt/sda1/datalog.csv

nano /mnt/sda1/datalog.py

#!/usr/bin/python

-- coding: utf-8 --

import sys
from datetime import datetime
data_rcv=sys.argv[1] +"," + sys.argv[2] +"," + str(datetime.now())
f = open('/mnt/sda1/datalog.csv','a')
f.write(data_rcv)
f.write('\n')
f.close()

chmod 755 /mnt/sda1/datalog.py
Testing:
/mnt/sda1/datalog.py '1' '2'
#include <Process.h>

void setup() {
// Initialize Bridge
Bridge.begin();
}

void loop() {
Process p;
p.begin("/mnt/sda1/datalog.py");
p.addParameter("1");
p.addParameter("2");
p.run();
delay(10000);
}

I'm sorry sonnyyu, could you tell how to write those code in arduino IDE step by step ?

sonnyyu · June 23, 2015, 11:46am

Using the command line for communication with SSH

A Beginner's Guide to SSH (Putty)

http://www.gamexe.net/other/beginner-guide-ssh/

The Beginner’s Guide to Nano, the Linux Command-Line Text Editor

enormousdreamer · June 23, 2015, 2:17pm

sonnyyu:
Using the command line for communication with SSH

http://www.arduino.cc/en/Tutorial/LinuxCLI

A Beginner's Guide to SSH (Putty)

http://www.gamexe.net/other/beginner-guide-ssh/

The Beginner’s Guide to Nano, the Linux Command-Line Text Editor

http://www.howtogeek.com/howto/42980/the-beginners-guide-to-nano-the-linux-command-line-text-editor/

Thanks, I'll have some time to study more about SSH.

Well, somehow I manage to find another solution by editing my previous code. I found that by calling the getTimeStamp() before the calling it again to add the variable "Data", give me the proper data logging with current timeStamp.

void loop() {
  getTimeStamp(); // BY ADDING THIS CODE
  Data += getTimeStamp();
  Data += ",";
  Data += "korewa-korewa";
  Serial.print(Data);
  Serial.println(" - Before writing data to SD Card");
  File dataFile = FileSystem.open("/mnt/sd/timetest1.txt", FILE_APPEND);

  // if the file is available, write to it:
  if (dataFile) {
    dataFile.println(Data);
    dataFile.close();
    // print to the serial port too:
    Serial.print(Data);
    Serial.println(" - After writing data to SD Card");
  }  
  // if the file isn't open, pop up an error:
  else {
    Serial.println("error opening datalog.txt");
  } 
  Data += "";
  delay(1000);
}

I assume the getTimeStamp() might not be ready after the looping. However, by calling it before adding the data to the variable might make the getTimeStamp() is ready to work.

Topic		Replies	Views
Bridge and getTimeStamp issue Yún	11	5093	May 6, 2021
Arduino YUN strange behaviour Yún	7	1724	May 6, 2021
Trouble getting time/date stamp onto data logging file Programming	2	1809	May 5, 2021
[Solved] SD Datalogger dataFile.close() issue Yún	2	1825	May 6, 2021
Error opening datalog using Arduino Yun and AltSoftSerial Yún	4	1167	May 6, 2021

Data Logging Problem - TimeStamp Stop Working After Write Data to SD Card

-- coding: utf-8 --

Related topics