Pages: [1]   Go Down
Author Topic: Logging using NTP as Time Stamp  (Read 1075 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 2
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I am currently investigating an easy way to record AGC voltages from a radio network for my employer an discovered Arduino's. As I am new to Arduino's I am in need of help. As can be seen by my code I am using a local timeserver on our LAN to get the time for a timestamp of the voltages. The voltages is then recorded as dBm values.

My problems is as follows:
I having difficulty logging to multiple files. I have looked at a few samples like adafruit's light and temp logger which logs to multiple files, but I cannot get my SD card initialized.
My second is how to handle the ethernet and SD access, as each log needs a timestamp(ethernet) and then writes the log to file(SD), without any SPI clash.
If I can get the above working I then need to access the above created logs via a webserver.

Any help?

Code:
#include <SPI.h>         
#include <Ethernet.h>
#include <SD.h>
#include <Time.h>
#include <Udp.h>

#define TIME_MSG_LEN  11   // time sync to PC is HEADER followed by unix time_t as ten ascii digits
#define TIME_HEADER  'T'   // Header tag for serial time sync message
#define TIME_REQUEST  7    // ASCII bell character requests a time sync message


const int chipSelect = 4;

int sensorPin0 = A0;// select the input pin for the potentiometer
int sensorPin1 = A1;
int ledPin = 13;      // select the pin for the LED
int sensorValue0 = 0;  // variable to store the value coming from the sensor
int sensorValue1 = 0;

// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = { 
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = {   XXX,XXX,XXX,XXX };

unsigned int localPort = 8080;      // local port to listen for UDP packets

byte timeServer[] = {
  XXX,XXX,XXX,XXX}; // timeserver on LAN
 
 
const int NTP_PACKET_SIZE= 48; // NTP time stamp is in the first 48 bytes of the message

byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing packets



void setup()
{
 // start Ethernet and UDP
  Ethernet.begin(mac,ip);
  Udp.begin(localPort);
 
  Serial.begin(9600);
  Serial.print("Initializing SD card...");
 
 
  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    // don't do anything more:
    return;
  }
  Serial.println("card initialized.");
  Serial.println("Waiting for sync message");
  delay(5000);
}


void loop()
{
   sendNTPpacket(timeServer); // send an NTP packet to a time server

    // wait to see if a reply is available
  delay(5000); 
  if ( Udp.available() ) { 
    Udp.readPacket(packetBuffer,NTP_PACKET_SIZE);  // read the packet into the buffer

    //the timestamp starts at byte 40 of the received packet and is four bytes,
    // or two words, long. First, esxtract the two words:

    unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
    unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]); 
    // combine the four bytes (two words) into a long integer
    // this is NTP time (seconds since Jan 1 1900):
    unsigned long secsSince1900 = highWord << 16 | lowWord; 
    Serial.print("Seconds since Jan 1 1900 = " );
    Serial.println(secsSince1900);               

    // now convert NTP time into everyday time:
    Serial.print("Unix time = ");
    // Unix time starts on Jan 1 1970. In seconds, that's 2208988800:
    const unsigned long seventyYears = 2208988800UL;     
    // subtract seventy years:
    unsigned long epoch = secsSince1900 - seventyYears; 
    // print Unix time:
    Serial.println(epoch);
    setTime(epoch);
       


    // print the hour, minute and second:
    Serial.print("The UTC time is ");       // UTC is the time at Greenwich Meridian (GMT)
    //Serial.println(epoch % 31556926 + 1970);
    Serial.print((epoch  % 86400L) / 3600); // print the hour (86400 equals secs per day)
    Serial.print(':'); 
    Serial.print((epoch  % 3600) / 60); // print the minute (3600 equals secs per minute)
    Serial.print(':');
    Serial.println(epoch %60); // print the second
  }
 
 }

// send an NTP request to the time server at the given address
unsigned long sendNTPpacket(byte *address)
 {
  // set all bytes in the buffer to 0
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  // Initialize values needed to form NTP request
  // (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12]  = 49;
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;

  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp:    
  Udp.sendPacket( packetBuffer,NTP_PACKET_SIZE,  address, 123); //NTP requests are to port 123

  pinMode (10,OUTPUT);
  digitalWrite(10,HIGH);
     
  {
    digitalWrite(13,timeStatus() == timeSet); // on if synced, off if needs refresh 
 
  }

 // read the value from the sensors:
  sensorValue0 = analogRead(sensorPin0);
  sensorValue1 = analogRead(sensorPin1); 
  // turn the ledPin on
  digitalWrite(ledPin, HIGH); 
  // stop the program for <sensorValue> milliseconds:
  delay(sensorValue0);
  delay(sensorValue1);
  // turn the ledPin off:       
  digitalWrite(ledPin, LOW);   
  // stop the program for for <sensorValue> milliseconds:
  delay(sensorValue0);
  delay(sensorValue1);
 
   // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  File dataFile = SD.open("datalog.csv", FILE_WRITE);

  // if the file is available, write to it:
  if (dataFile) {
    dataFile.print("TX Level =, ");
    dataFile.print((sensorValue0/19.6)+6.5);
    dataFile.print(",  RX Level =, ");
    dataFile.print((sensorValue1/9.8)-78.5);
    dataFile.print(" , ");
    dataFile.print(day());
    dataFile.print("/");
    dataFile.print(month());
    dataFile.print("/");
    dataFile.print(year());
    dataFile.print("  ");
    dataFile.print(hour());
    dataFile.print(" : ");
    dataFile.print(minute());
    dataFile.print(" : ");
    dataFile.print(second());
    dataFile.println(); 
    dataFile.close();
    // print to the serial port too:
    Serial.print("TX Level =, " );                       
    Serial.print((sensorValue0/19.6)+6.5);   
    Serial.print(",  RX Level =, ");
    Serial.println((sensorValue1/9.8)-78.5);
    Serial.print(" , ");
    Serial.print(day());
    Serial.print("/");
    Serial.print(month());
    Serial.print("/");
    Serial.print(year());
    Serial.print("  ");
    Serial.print(hour());
    Serial.print(" : ");
    Serial.print(minute());
    Serial.print(" : ");
    Serial.print(second());
    Serial.println();   
  } 
  // if the file isn't open, pop up an error:
  else {
    Serial.println("error opening file.");
  }
delay(5000);
 
}


Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 220
Posts: 13846
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


You better use a real time clock that get synchronized with NTP so you don't need to use NTP every logentry. Makes life a bit easier.

See my playground article - http://www.arduino.cc/playground/Main/DS1307OfTheLogshieldByMeansOfNTP

Note that a RTC like the 1307 is not as accurate as using NTP, but it might be good enough.

If you don't want to use an RTC you could get the NTP stamp in setup() and just add millis() to that value to get a reasonable timestamp. As millis() overflows every 49 days or so you need to reread the NTP at least every overflow but once a day could be enough.





Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Offline Offline
Newbie
*
Karma: 0
Posts: 2
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Thanks for your help.
I also thought of using a RTC but not only will it take up space that I dont have but it will also occupy some of the pins that I need to use.
I will look in to using millis () as you suggested. The time stamp does not need to be that accurate as my readings will be taken every 10 or so seconds.

The multiple logged files is a big concern for me. If I can around this problem I can start rolling out the project as the rest of my problems are "nice to have's".

Again many thanks
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 220
Posts: 13846
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The SD library performes better if you write large chunks of data. writing 3 bytes to a files every 10 seconds const more time than writing 30 bytes to a file every 100 seconds. If memory permits create a small cache to hold multiple samples e.g. for 1 minute.

Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 220
Posts: 13846
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

You should split up the program in some more functions

some pseudo code (will NOT compile)
Code:
// local vars here

void setup()
{

  long timestamp = GetNTPTimeStamp();  // can be converted to day month year H:m:s  This function contains all NTP stuff, it returns only the whole seconds part
  long TSmillis = millis();  // the millis when timestamp was made

  Serial.begin(115200);  // as fast as possible ...
  // .. other setup stuff
}

void loop()
{
  // MAKE MEASUREMENTS
  rawValue = analogRead(A0); // whatever

  // PROCESS THEM
  meaningfullValue = f(rawValue);  // convert to volts or ??

  // LOG THEM
  char buffer[100];
  sprintf(buffer, "......", values);
  WriteToLog(filename, buffer);
}

int WriteToLog(char * fname, char* buf)
{
  // OPEN FILE
  // if error return error code

  // WRITE buf to FILE
 
  // CLOS FILE
}

long GetNTPTimeStamp()
{
  // all NTP code here

  return seconds;
}

The above code will not compile but gives you some sight on the structure...

Hopes this helps
Rob

Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Pages: [1]   Go Up
Jump to: