Serial clock data

Hello all, ive got a little stuck with this one.

I have a digital master clock, its basically a NTP time server mostly used for syncing computer systems and timestamping etc.
What I am aiming to do us use its RS232 serial output and build a interface so that it can drive a relay at certain intervals.
I have got as far as getting the UNO to receive the incoming data and print it to the serial monitor.

What I am struggling with is how to use this data. Its a block of 15 bytes received every second containing seconds, minutes, hours, date etc etc.
I only want to use the seconds data for now to drive a relay every 30 seconds but hopefulyy add more function later.

I need to assign each byte of data a name so that I can call an IF function eg // if byte two is 48 and byte three is 51, operate relay for some sort of period of time. (bytes in DEC).

Is anyone able to advise? Ive spent hours looking into this and cant seem to find any info on it.

Thanks in anticipation

I have got as far as getting the UNO to receive the incoming data and print it to the serial monitor.

Please post the code you are using to read the message and the exact format of the 15 byte message.

Are the field widths, for example seconds, always two digits with a leading 0, or will they be one digit for values less than 10?

Use the Time library. It has lots of date-time functions. I use it is some projects that have no clock at all.

That library is usually used with RTC chips but it can accept input from anywhere.

Hi, below is the simple sketch im using to receive the data.

int incomingByte = 0;   // for incoming serial data

void setup() {
        Serial.begin(9600);     // opens serial port, sets data rate to 9600 bps
}

void loop() {

        // send data only when you receive data:
        if (Serial.available() > 0) {
                // read the incoming byte:
                incomingByte = Serial.read();

                // say what you got:
                Serial.print("I received: ");
                Serial.println(incomingByte, DEC);
        }
}

An example of the format returned is this, but remember this is time data sent every second so it all increments

bytes 1-15 are

2 start transmission
52 seconds digit 1
51 seconds digit 2
50 minutes digit 1
48 minutes digit 2
48 hours digit 1
48 hours digit 2
49 day of month digit 1
48 day of month digit 2
49 month digit 1
48 month digit 2
57 year digit 1
57 year digit 2
48 status (synchronised to dcf 77 or not)
3 end transmission

The first thing to review is Robin2's tutorial on receiving Serial input. Serial Input Basics.

You can use the receive with Start/End markers example. Your start marker is 0x02 and your end marker is 0x03.

const byte numChars = 32;
char receivedChars[numChars];
boolean newData = false;

void setup() {

  Serial.begin(9600);
  Serial.println("Arduino is ready --");
}

void loop() {
  recvWithStartEndMarkers();
  showNewData();  
}

void recvWithStartEndMarkers() {
  static boolean recvInProgress = false;
  static byte ndx = 0;
  char startMarker = 0x02;
  char endMarker = 0x03;
  char rc;
  
  if (Serial.available() > 0) {
    rc = Serial.read();

    if (recvInProgress == true) {
      if (rc != endMarker) {
        receivedChars[ndx] = rc;
        ndx++;
        if (ndx >= numChars) {
          ndx = numChars - 1;
        }
      }
      else {
        receivedChars[ndx] = '\0'; // terminate the string
        recvInProgress = false;
        ndx = 0;
        newData = true;
      }
    }

    else if (rc == startMarker) {
      recvInProgress = true;
    }
  }
}

void showNewData() {
  if (newData == true) {
    Serial.print("This just in ... ");
    Serial.println(receivedChars);
    newData = false;
  }
}

When you have this data on the Arduino, there are several things you can do with it.
MarkT's suggestion to set the time library is good. You can sync the internal arduino clock with the received data at any interval you want. Or else, you can just pull out the seconds from the received data and convert the chars to an integer number.

You will need to implement something like this with the seconds value and a boolean control variable, whether you use the incoming seconds data from the received data or use the time library.

if (seconds ==30 && actionTriggered == false)
{
//trigger relay control action
actionTriggered = true;
}

Then when the relay control action is complete set actionTriggered back to false.

thanks for this chaps, using the above sketch helps as the data is in a more readable format, even though it is coming through backwards.
Unfortunately though, im still not seeing a way to use this data for what i need to use it for.
I need to call a function to extract particular bytes but the data is simply coming through in a block of numbers.
How can i separate it into useful things to enable parts of it to be used?
i read the thread tutorial on receiving serial data several times, but this didnt really help with what i'm actually trying to do?
I've never tried to actually receive serial data before in order to make arduino do something so this concept is very new to me.
I really appreciate your help.
steve.

Good progress. You now have a reliable way to read the data into the Arduino. The byte order can certainly be accommodated.

I need to call a function to extract particular bytes but the data is simply coming through in a block of numbers.
How can i separate it into useful things to enable parts of it to be used?

What you need to realize is that the incoming data is now stored on the Arduino in a null terminated character array and each element has an index. The first index is 0.

In Robin2's example code that array is called receivedChars[]. If the characters between the start and endmarkers were 87654321, then the value of receivedChars[0] is 8 and receivedChars[7] is 1. The underlying value stored in the array element is the binary of the ascii value, but the code supplied prints them out as the characters.

You now have to decide which array elements you want to pull out, and what you want to do with them.

I understand you want to convert then from chars/ascii value to actual numbers. There are several ways to do this, but since you are mostly dealing with two digit numbers its easy and simple to subtract 48 (or char '0') from the ascii value and you have the digit.

So, if the data were 87654321 and you wanted to turn elements 6 with byte/ascii value 50 and element 7 with byte/ascii value 49 into a two digit number 21 then you will have

byte myNewNumber = (receivedChars[6] - '0')*10 +receivedChars[7] -'0';

Play around with extracting numerical values from the array by index, and get back with any more questions.

Hi, I dont need to convert the ascii values into actual numbers, simply use the receivedChars 1 and 0 (in that order because the data string is running backwards(although in real terms it probably dont actually matter)) and call it to pull a relay in every thirty seconds. I'm probably missing something that is staring me straight in the face, but i'm having great difficulty finding a way of extracting the numerical values.
Just when I think i'm beginning to understand how all this works, i now understand i'm fairly clueless in many areas.
Is there anything i can add into the above code to simply prove it works before i write something more specific? if so, where exactly is best to insert it?
I might have to admit defeat otherwise until i can find the time to better educate myself on this things.
I suppose its the typical "trying to run before i can walk" scenario

Is there anything i can add into the above code to simply prove it works before i write something more specific? if so, where exactly is best to insert it?

but i'm having great difficulty finding a way of extracting the numerical values.

OK. Lets work through this.

  1. Detecting when when receivedChars[1] == '3' and receivedChars[0] == '0'?

The test for the time match needs to be when newData == true.

If you don't want to convert the chars to the actual number 30 like in the previous code, then you can use a compound conditional statement like

 if(receivedChars[1] == '3' && receivedChars[0] == '0')
{
  //do what needs to be done at a time match
}

The && is the syntax for "logical and" and it means both conditions need to be true for the statement to be true. The Arduino ide, even makes it easier, and you could use the word "and" if the && confuses you.

The "devil is in the details" of this code is when to set the newData flag back to false, and what do you want to do on the time match.

Depending on the relay action, there are different ways to go and different times to set the newData flag back to false. My preferred method would be to use a new boolean flag variable runRelayCycle. I'd set it true on the match and then set newData = false.

if(receivedChars[1] == '3' && receivedChars[0] == '0' && runRelayCycle ==false)
{
runRelayCycle = true;
newData = false;
}
  1. Your next issue is what to do with the relay when the time match occurs.

All the relay timing and setting the runRelayCycle condition back to false, so that you can act on the next time match, will be in a new conditional statement.

Most important is how long does the relay need to be on, and if you need to turn it on and off with non blocking techniques using a millis() timer instead of delay()?

There are some details here if to make the timing non blocking but lets deal with that when you get the time match part of the code under your belt. Conceptually, the relay code will be like this

if(runRelayCycle ==true)
{
   //turn relay On
   //start timing
   //turn relay Off
   //runRelayCycle = false
}

See if you can come back with some code which implements the match on 30 seconds, and we'll go from there.

Hi, thank you again for you patience, I am now starting to understand.
I now have the seconds printing to serial monitor and have had output from pin 5 on the 30th second. Ive also added another part to give me output on 00 seconds, so now I have pin 5 operating ever half-minute as intended. Now, the code for this (as expected) gives Pin 5 'high' for one second. It could really do with being in the order of 500 milliseconds, maybe a little less.
I've had issues with the 'delay' function before, and i suppose its far from suitable in this particular application.
what do you think so far?
Kind regards, Steve

If there is absolutely nothing else useful that the Arduino can do during the delay, then use delay().

Forgot to add the code i currently have----

const byte numChars = 32;
char receivedChars[numChars];
boolean newData = false;
int relayPin = 5;


void setup() {

  Serial.begin(9600);
  Serial.println("Arduino is ready --");
}

void loop() {
  recvWithStartEndMarkers();
  showNewData();  
}

void recvWithStartEndMarkers() {
  static boolean recvInProgress = false;
  static byte ndx = 0;
  char startMarker = 0x02;
  char endMarker = 0x03;
  char rc;
  
  if (Serial.available() > 0) {
    rc = Serial.read();

    if (recvInProgress == true) {
      if (rc != endMarker) {
        receivedChars[ndx] = rc;
        ndx++;
        if (ndx >= numChars) {
          ndx = numChars - 1;
        }
      }
      else {
        receivedChars[ndx] = '\0'; // terminate the string
        recvInProgress = false;
        ndx = 0;
        newData = true;
      }
    }

    else if (rc == startMarker) {
      recvInProgress = true;
    }
  }
}

void showNewData() {
  if (newData == true) {
    if((receivedChars[1] == '3' && receivedChars[0] == '0')||(receivedChars[1] == '0' && receivedChars[0] == '0'))
{
  digitalWrite (relayPin,HIGH);//do what needs to be done at a time match
}
else digitalWrite(relayPin,LOW);
    

    Serial.print("Seconds now ... ");
    Serial.print(receivedChars[1]); 
    Serial.println(receivedChars[0]);
       
    newData = false;
  }
}

MorganS:
If there is absolutely nothing else useful that the Arduino can do during the delay, then use delay().

as it stands at the moment, I 'could' use delay, but as I build on this project I think it will cause issues in the future.
The 30 second relay output is to drive electro mechanical slave clocks and i would eventually like automatic advance/retard feature when summer/winter time changes occur.
Again, its something else i'm struggling to find a solution for, but clearly I need to understand more about the basics first.

I have a digital master clock, its basically a NTP time server mostly used for syncing computer systems and timestamping etc.
What I am aiming to do us use its RS232 serial output and build a interface so that it can drive a relay at certain intervals.

The 30 second relay output is to drive electro mechanical slave clocks and i would eventually like automatic advance/retard feature when summer/winter time changes occur.

I would like to hear more about the overall design, hardware and software, of your project. Using the Time Library possibly along with TimeZone.h to manage the summer/winter changes, may be the way to go. The Arduino time will be synchronized with an NTP server at intervals.

If you want to run the relay for 500 ms with a non blocking timer, the code can look like this:

const byte numChars = 32;
char receivedChars[numChars];
boolean newData = false;
int relayPin = 5;

boolean relayRunning = false;
unsigned long relayStartTime;

void setup()
{
  Serial.begin(9600);
  Serial.println("Arduino is ready --");
}

void loop()
{
  recvWithStartEndMarkers();
  processNewData();
  runRelayCycle();
}

void recvWithStartEndMarkers() {
  static boolean recvInProgress = false;
  static byte ndx = 0;
  char startMarker = 0x02;
  char endMarker = 0x03;
  char rc;

  if (Serial.available() > 0) {
    rc = Serial.read();

    if (recvInProgress == true) {
      if (rc != endMarker) {
        receivedChars[ndx] = rc;
        ndx++;
        if (ndx >= numChars) {
          ndx = numChars - 1;
        }
      }
      else {
        receivedChars[ndx] = '\0'; // terminate the string
        recvInProgress = false;
        ndx = 0;
        newData = true;
      }
    }

    else if (rc == startMarker) {
      recvInProgress = true;
    }
  }
}

void processNewData()
{
  if (newData == true)
  {
    if ((receivedChars[1] == '3' && receivedChars[0] == '0') || (receivedChars[1] == '0' && receivedChars[0] == '0'))
    {
      if (relayRunning == false)
      {
        digitalWrite (relayPin, HIGH);
        relayRunning = true;
        relayStartTime = millis();
      }
    }
    Serial.print("Seconds now ... ");
    Serial.print(receivedChars[1]);
    Serial.println(receivedChars[0]);

    newData = false;
  }
}

void runRelayCycle()
{
  const unsigned long relayInterval = 500;
  if (relayRunning == true && millis() - relayStartTime >= relayInterval)
  {
    digitalWrite(relayPin, LOW);
    relayRunning = false;
  }
}

What i have is a 'Wharton Electronics 4850 time source." on the following link can be found the operators manual for this device. Check www.manualslib.com.
As it is able to do various things, it was thought that this device can also run electro-mechanical slave clocks, this function currently carried out by a pendulum master clock requiring constant adjustment. Although there is a relay output available on this model, it does not cater for half-minute operation.
Because i am also looking for more functionality, I surmised that reading the serial data from its rs232 db9 socket would be the way forwards. I currently have the very bare bones to enable it to correctly read the serial data, and provide a relay output of around 400ms duration every half-minute.
What i would eventually like to do is be able (using a keypad/a few buttons and a screen) is to be able to program the time that the slave clocks are showing. once we have programmed this info, allow arduino to either advance or retard slave clocks untill the proper time is shown. i think i need a way of using the master clock to drive the slave output which itself drives another clock so we can then compare what they are doing. when master clock jumps forward one hour at the end of this month, the two compare times and the latter advances itself and the slave clocks connected to it by one hour. i hope i am making some sort of sense with this.
I tidied up the code a little and so far have this. i also have the half min output driving a slave clock via a transistor and relay. The one slightly odd thing is that after an hour or so, nothing is printed to serial monitor without restarting the IDE and re-uploading the sketch. a bug somewhere perhaps, or do you think its something to do with the code? despite this, relay function carries on as normal.

const byte numChars = 32;
char receivedChars[numChars];
boolean newData = false;
int relayPin = 8;


void setup() {

  Serial.begin(9600);
  Serial.println("Arduino is ready --");
  pinMode(relayPin, OUTPUT);
}

void loop() {
  recvWithStartEndMarkers();
  showNewData();  
}

void recvWithStartEndMarkers() {
  static boolean recvInProgress = false;
  static byte ndx = 0;
  char startMarker = 0x02;
  char endMarker = 0x03;
  char rc;
  
  if (Serial.available() > 0) {
    rc = Serial.read();

    if (recvInProgress == true) {
      if (rc != endMarker) {
        receivedChars[ndx] = rc;
        ndx++;
        if (ndx >= numChars) {
          ndx = numChars - 1;
        }
      }
      else {
        receivedChars[ndx] = '\0'; // terminate the string
        recvInProgress = false;
        ndx = 0;
        newData = true;
      }
    }

    else if (rc == startMarker) {
      recvInProgress = true;
    }
  }
}

void showNewData() {
  if (newData == true) {
    if((receivedChars[1] == '3' && receivedChars[0] == '0')||(receivedChars[1] == '0' && receivedChars[0] == '0'))
{
  digitalWrite (relayPin,HIGH);
  delay(400);//do what needs to be done at a time match
  digitalWrite (relayPin,LOW);
}

    

    Serial.print("Seconds now ... ");
    Serial.print(receivedChars[1]); 
    Serial.println(receivedChars[0]);
       
    newData = false;
  }
}

Although there is a relay output available on this model, it does not cater for half-minute operation.

The relay output can be programmed for 5 second operation. I think that counting closures with an interrupt and triggering the slave clock pulse every 6 interrupts would be an alternative to working with the serial ascii time stamp output.

You can use the serial time stamp data for your enhanced functionality, and system wide time adjustments, but I'm not sure if generating the regular slave pulse from it is the best approach.

What i would eventually like to do is be able (using a keypad/a few buttons and a screen) is to be able to program the time that the slave clocks are showing. once we have programmed this info, allow arduino to either advance or retard slave clocks untill the proper time is shown. i think i need a way of using the master clock to drive the slave output which itself drives another clock so we can then compare what they are doing. when master clock jumps forward one hour at the end of this month, the two compare times and the latter advances itself and the slave clocks connected to it by one hour.

I would consider using the Time Library and Time Zone library to manage the time change rules for summer time or other global settings. When the master clock jumps forward or backwards, the Arduino will be aware of this, and issue the correct number of pulses to adjust the slaves.

Developing a user interface to have the Arduino issue pulses or withhold them for time adjustments or setting time seems relatively simple.

I guess, I would consider using the relay output for the regular slave pulses, and the Serial time stamps for synchronizing the Time Library to the 4850 and managing the time change rules.

What I don't understand is how you recover from a power failure. Will the 4850 resync to its control when power returns? Is it on a battery backup and will keep operating and not loose correct time? How will you know what time the slaves stopped, and how much to adjust them when power comes back?

The only issue I see is the 4850's on board relay is part of the PCB and i'm concerned that if it ever failed it wouldnt be a simple task to change it.
With regards to power failures, the current system (using the pendulum driven master clock) is run from a 24v battery kept on float charge. The 4850 can be powered by a 12v supply so the intention would be to run it from the battery via a dc-dc converter instead of from the mains.
The other issue is that the slave clocks in this system are current driven and so are wired in series. A fault in any part of the system stops everything hence the requirement to be able to program the slave clocks indicated time to allow arduino to make a comparison between that and the time the 4850 is providing when any issues have been resolved. Now a properly installed system rarely gives such trouble, its usually a problem when somebody decorates a room and removes the clock from the wall not knowing its part of a series system.
The other potential issue i see in using the relay for the main clock drive function is how does arduino make such a comparison with the time? If the 4850 timekeeping drifted a long way such might be the case with a loss of DCF 77 signal, then it suddenly corrected, wouldnt there be a risk of either missing a 5 sec pulse or gaining one somewhere down the line?
I'm definately re-enthused with this project thanks to your support.

It sounds like you have good rationale to manage the slave pulses from the serial data, and I'm sure you can make that work. You may wish to increase the baud rate of the 4850. It can't hurt to have less message processing time. I also see no downside of placing the relay on the non blocking millis() timer. With increased functionality you will want the system to be responsive to inputs.

The battery backup sounds good, and the 4850 sounds very accurate even when it is not receiving DCF77 synchronization. Do you know if the unit provides the time change warnings and other features of the DCF77 system?

It sounds like once you implement the Arduino control of the 30 second pulse you will have replaced the pendulum master.

You know how you manage the existing system, so you have an idea of where you want to go for time change adjustments and unintended errors. If at present, a clock has been removed, and the system gets disconnected from the pendulum master time, how do you determine the discrepancy? How do you make adjustments to the slaves?

I think you will need a slave clock at the control station for comparison with the master. You might be able to use pulse signal feedback from the parallel loop back into the Arduino to make it a slave clock and you will be able to detect the difference from it and the 4850 time and make automatic adjustments.

You will need to give some thought to the human interface to the Arduino, and whether that should be a laptop, phone with bluetooth, or buttons and encoders. Menu design is not trivial.

You are lucky to have the 4850, as most of the postings about Arduino's and "atomic" or "radio controlled" clocks are focused on decoding the DCF77 or WWVB(NIST) broadcasts. There is plenty to be found with Mr Google.

Good luck with your project.

currently, if the clock system is stopped we make sure all slave clocks are showing the correct time, swith the master to 1 sec output to advance them all to the correct time.
I have also implimented your code using the millis function and seems to have stopped the serial monitor from playing up.
One problem i have come across is, the 4850 only needs to be a second fast, gives pulse at top of minute, dcf resets it back again, we end up with two pulses. Slave clock now 30 seconds fast. another reason why arduino needs to know what the clocks are saying. or is there a way of preventing pulses happening within say, 15 seconds of each other.

As for auto summer and winter correction, it occured to me that byte 14 is the status byte.
Bit 1 is 1 for summer time, 0 for winter time.
Can we extract this bit easily?
My thoughts were //if byte 14, bit 1 goes to 1, provide 124 pulses at 1 second intervals. if byte 14, bit 1 goes to zero, stop pulse output for one hour.

this could possibly eliminate any form of interface.

As for auto summer and winter correction, it occured to me that byte 14 is the status byte.
Bit 1 is 1 for summer time, 0 for winter time.
Can we extract this bit easily?

Absolutely. Use the same reading by index technique you use to extract the seconds.

You will need a simple routine to compare the value of the bit to the last read value of the bit, and if there was a change from 0 to 1 set forward one hour, if the change was from 1 to 0 set back by stopping pulses for an hour, or advance 23 hours. It's standard state change code where the last reading takes on the value of the current reading, and the next current reading is compared to the last.

One problem i have come across is, the 4850 only needs to be a second fast, gives pulse at top of minute, dcf resets it back again, we end up with two pulses. Slave clock now 30 seconds fast. another reason why arduino needs to know what the clocks are saying. or is there a way of preventing pulses happening within say, 15 seconds of each other.

I don't understand this. The 4850 is supposed to be very accurate on its internal oscillator. Is loss of synch with dcf77 a common event?

I think your idea of checking if you read twice at 00 seconds less than a minute apart is a good one. There are probably several ways to do this.

One possibility would be to modify this code which turns the relay on

if (relayRunning == false)
      {
        digitalWrite (relayPin, HIGH);
        relayRunning = true;
        relayStartTime = millis();
      }

It could change to this using a new variable lastRelayStartTime

if (relayRunning == false)
     {
       relayStartTime = millis();
       if(relayStartTime - lastRelayStartTime >= 10000) //some lock out period
        {
          digitalWrite (relayPin, HIGH);
          relayRunning = true;
          lastRelayStartTime = relayStartTime;
         }
      
       
     }

EDIT: Changed position of assignment of lastRealaySTartTime to relayStartTime. Previous error.