Guidance needed on processing serial data

Hello all.

I'm working on a project reading incoming serial data from a remote control keypad device. The data sent is always 3 bytes of information with the first byte always the same. (0xFE) It only sends data when a key is pressed, so there is no constant stream of data, or a 'no key is being pressed' packet.

I've been able to read the data and decode it just fine for the most part but have run into a problem with the fact that it constantly sends a new packet of 3 bytes as fast as the connection (9600 baud) will allow as long as a key is held down. I do not know how to take this repeated information and use it as one event, as opposed to dozens of repeated events per second.

Ex: while a key is held down on the device it will send 0xFE FF FF over and over again and i want an led to remain turned on constantly as long as the button is held down.

Adding to my trouble is the fact that either the device or my code reads a random zero value from time to time even when the key is held down.

I have searched to the best of my ability for an example of how to deal with this, and so far have come up short. Any help would be greatly appreciated.

I'll be happy to provide whatever further information is needed. My current sketch is a mess of other functions for deciphering the incoming data as i need, but is mostly running as follows, in pseudo code:

if serial available > 2 (if there is at least one full 3 byte command) if first byte = 0xFE, if not read the next byte read second byte, shift 8 bits left read third byte, add to shifted second byte Store 16 bit number to int variable check 16 bit var against conditions, do things if the number matches.

What I need to figure out is how to make it keep doing the same function as long as the incoming data remains the same, then stop when no new data, or different data comes in.

Thanks, JGL

pseudo code if new int variable is the same as last int variable, do nothing. In other words, only accept it as new data if new int variable is different from last int variable. If you need to accept the same data twice in a row, use a time-out. if new int variable is the same as last int variable, and 5(?) seconds hasn't passed since the last data was received, do nothing. In other words, only accept it as new data if new int variable is different from last int variable [u]or[/u] 5(?) seconds has passed since the last data was received.

You could also add an alarm if the user leans on the key for more than a certain amount of time.

if serial available > 2 (if there is at least one full 3 byte command) if first byte = 0xFE, if not read the next byte read second byte, shift 8 bits left read third byte, add to shifted second byte Store 16 bit number to int variable check 16 bit var against conditions, do things if the number matches.

or

if serial available > 2 (if there is at least one full 3 byte command) readBytes ( buffer,3) read three bytes to local char buffer with optional timeout than analyze buffer if first byte = 0xFE, if not read the next byte - maybe you mean "if == 0xFE "

check if second and third bytes match test condition and / or create 16 bit variable

of course both ways assume you receive 3 bytes always , not much for error checking

Good luck

Thank you for the input so far.

The problem I'm having is not with getting the data in the first place but only with the fact that the keypad device constantly refreshes, sending repeated data as long as a button is held down. On some of the keys I need to turn on a function as long as the button is held down. on others I need a one-shot function to turn on and off one time no matter how long the button is pressed and on still others I need it to read each packet of data as a new event.

In addition I am plagued by the fact that even while holding a button down, once in a while ( 8-12 cycles) the serial will read that there is no data present, causing the data to change back and forth from pressed to not and back again.

If we assume I have a 16 bit int that is refreshed on every cycle of the main loop, how can I say if it has not changed 'button 1 = on' and exclude any reads of nothing on the serial? Perhaps there is a way I can look for 5-10 repeated reads of nothing no data) and use that as a marker for when a button is released?

I'm sure the solution is easier than I'm thinking, but it's giving me fits at this point.

Thanks again. JGL

while holding a button down, once in a while ( 8-12 cycles) the serial will read that there is no data present, causing the data to change back and forth from pressed to not

You shouldn’t try to read the data, if no data are present, and implement a timeout to determine when “no button is pressed”.

Post your code, using code tags ("</>" button).

Well, yes, I’m not reading the serial data when there is none, but what I mean is the int created from the data will read a zero. I need a way to throw out the occasional cycle where no new data was present.

 void readSerialData() {
      if (Serial1.available() > 2) { // check if there is enough data on serial buffer for a command.
          inByte = Serial1.read();  // read the serial data 
          if (inByte = 0xFE) { // Check if the data is a vaild command starting with 0xFE
              //systemHALTclear();  // Clear HALT condition if any new commands are sent from remote.
              byteTwo = Serial1.read();  // Read next byte of serial data as byteTwo
              byteThree = Serial1.read();  // Read next byte of serial data as byteThree
          }
          else{
              checkNextByte(); // If the data recived is not a valid command skp to next byte and try again.
          }
          
      byteTwoShifted = byteTwo<<8; // shift byteTwo 8 bits left.
      fullWord = (byteTwoShifted + byteThree); // Combine the bytes into one 16 bit int

Here I dissect the int ‘fullWord’ into the various parts I need for different functions ex:

commandTypeFirstCheck = fullWord>>11; // first check of type = most significnt 5 bits of fullWord.
          
          if (commandTypeFirstCheck < 24) { // if the 5 bits is < 24 it is a 2 bit commandType (0=functionA, 1=FunctionB, 2=FunctionC)
            commandType = fullWord>>14;  // set command type to two bit type.
            // addrsss setup for 7 bit device addresses
            commandAddressFirstShift = fullWord<<2;  // shift off left most 2 bits
            addressedDevice = commandAddressFirstShift>>9; // shift off all but 7 most significant bits.    
          }

this will set the vars I need elsewhere of commandType and addressedDevice.
The main loop finishes by reseting the fullWord to 0 before returning to the top to start over.

JGL

Whatever checkNextByte() does, I’m quite sure that that causes the problems. It’s probably an attempt to sync the stream. What does checkNextByte() do?

To make sure that your data is valid, I suggest that you change readSerialData() to return a flag that you can test; the flag indicates if valid data is received or not so you can ignore the data if needed.

bool readSerialData()
{
  static int prevFullWord = 0;
  bool isValid = false;

  if (Serial1.available() > 2)
  { // check if there is enough data on serial buffer for a command.
    inByte = Serial1.read();  // read the serial data
    if (inByte = 0xFE) { // Check if the data is a vaild command starting with 0xFE
      //systemHALTclear();  // Clear HALT condition if any new commands are sent from remote.
      byteTwo = Serial1.read();  // Read next byte of serial data as byteTwo
      byteThree = Serial1.read();  // Read next byte of serial data as byteThree
      isValid = true;
    }
    else
    {
      checkNextByte(); // If the data recived is not a valid command skp to next byte and try again.
    }

    if (isValid != true)
      return isValid;

    byteTwoShifted = byteTwo << 8; // shift byteTwo 8 bits left.
    fullWord = (byteTwoShifted + byteThree); // Combine the bytes into one 16 bit int

    if (prevFullWord != fullWord)
    {
      prevFullword = fullWord;
      // do whatever
    }
  }
  return isValid;
}

The above code also keeps track of the last received fullWord so you can compare it against the new one and do something in the case that it changed. You probably don’t want to ‘do whatever’ in the serial read, but without seeing the full code I have no idea where to add / change code.

Please post your complete code for better help; at this stage I (and probably others) can not follow the flow completely. At this moment the above code is just ideas.

this is the missing bit there.

void checkNextByte() {
      int trashData = 0;
      trashData = Serial1.read();
      Serial.println(" ");
      Serial.println("T R A S H   D A T A :");
      Serial.println(trashData, HEX);
      Serial.println(" ");
      delay(5000);
    }

in several hours of working on this so far this loop has never been called by the program, so I really don't know if it works as intended or not. I would notice a 5 second stop in the serial monitor as I have it printing the 'fullWord' every cycle. My problem is not in getting the 16 bit int reliably, but in what to do with it after I have it.

I'll see what I can do to clean up the code and post it in full tomorrow but at the moment it is quite the jumble with the serial monitor displaying information that likely won't make sense to anyone else.

the 16 bits in question represent 4 fields of information with a device type that can be 2,4, or 5 bits, a device address that can be 4, 5, or 7 bits, a function of device field that is 2 bits, and a control over that function of 5 bits. the rest of my code is splitting the data, checking which of the various types it is, then it is checking to see if the remote is talking to one of two device types, and one of the two specific addressed for each of those types. Only if so does it go on to see the final 7 bits of what to actually go do with it's self.

None of this actually matters to the problem of determining if a button is being held down on the remote. I think I've got myself onto an idea now of counting cycles where there is no data, and using that as a way to tell the program that the button has actually been released and it is not just one of the fluke reads where there is no data for just one cycle. something like:

if fullWord == 0 noButtonThisCycle++ else noButtonThisCycle = 0

if noButtonThisCycle ==10 button is not pressed

I'll report back after work some time tomorrow.

GJL

Serial.read() returns 0 when no data is available or a 0 has been sent.

You could make a state machine that starts at state 0 and upon reading 0xFE goes to state 1 that reads and sets up the data high byte then to state 2 that reads the low byte. The state machine handles inputs differently depending on what came before. You can break any task down and run it in pieces with state machines.

The 2nd address in my sig space below has a good clear full tutorial on reading and processing serial data without blocking (blocking, why and how not to are explained in the 1st address tutorial) other tasks (like a status light and buttons) from running smoothly.

If you want informed help, post ALL the code. Snippets are completely useless.

Have a look at Serial Input Basics - simple reliable ways to receive data. There is also a parse example.

...R

I hope you have corrected this snippet , easy to overlook assignment from evaluation.

if (inByte = 0xFE) { // Check if the data is a vaild command starting with 0xFE //systemHALTclear(); // Clear HALT condition if any new commands are sent from remote. byteTwo = Serial1.read(); // Read next byte of serial data as byteTwo byteThree = Serial1.read(); // Read next byte of serial data as byteThree

I was able to solve my problem and figured from my experience on other types of forums it would be polite to let folks know as much, and how it was solved. Thanks to all that tried to help.

The most useful thoughts for me came in the following:

Henry_Best: pseudo code if new int variable is the same as last int variable, do nothing. In other words, only accept it as new data if new int variable is different from last int variable. If you need to accept the same data twice in a row, use a time-out. if new int variable is the same as last int variable, and 5(?) seconds hasn't passed since the last data was received, do nothing. In other words, only accept it as new data if new int variable is different from last int variable [u]or[/u]

I ended up using a count of how many times data read as 0 to determine when it was a true release of a button and as follows the code works for my needs.

fullWord = the data being received on each cycle of the main loop. currentCommandWord = the data that I will then use to test conditions further along in the code.

if (fullWord != 0) {       // if command is not 0 
      currentCommandWord = fullWord;    // current command = whatever data is present.
      zeroDataCheck = 0;  // zero check reset - data is not a zero
     }
     if (fullWord == 0) {  // if the data is a zero...
       if (lastFullWord == 0) {  // AND if the last data was also a zero
         zeroDataCheck++;  // add one to the zero counter
       }
     }
       if (zeroDataCheck >= 2) {  // if the zero counter is over 2 (may be ajusted but 2 seems to work right now)
         currentCommandWord = 0; // current command = zero:  no button is being pressed
       }
       if (zeroDataCheck >= 100) { // this keeps the zero check Var from rolling over and eliminates... 
         zeroDataCheck = 20;       // any chance of problems if no button is pressed for a while
       }
     lastFullWord = fullWord; // after data has been checked, reset last command recieved.

( edited the post to add a line of code I missed somehow when copy/pasting. ) JGL

I would notice a 5 second stop in the serial monitor as I have it printing the 'fullWord' every cycle.

Who said anything about a 5 second [u]stop[/u]? Use millis() to determine whether the time has elapsed and the rest of your code can continue undisturbed in the meantime.

I put the delay in intentionally for the express purpose of making sure I would see it if I got bad data. In this case it does exactly what I want it to do and will be removed once I am finished working on the project.

JGL