Manchester encoding algorithm and library

Hi guys,

I was just about to start a new topic about Manchester encoding when I noticed that some other poor fellow had written about this subject earlier today in the thread: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1198895890.

My question is whether or not there is a library for Arduino which lets me Manchester encode bits that are to be shipped through the SoftwareSerial, but from the other thread I can see that there probably is not.

I would be very interested in somehow helping with implementing a Manchester encoding algorithm and library in Arduino, but I'm probably not the main man to implement it, taking skill level/free resources into account.

I hope that we can put Manchester encoding on the roadmap as I think it could be useful for quite a few people..

I read sparkfuns datasheets on their xmit and receive pair and did NOT see any mention of that. As best I can tell you can shiftout data and pulse it in.

Contact them and see if they have an example project for the arduino. :slight_smile:

I read sparkfuns datasheets on their xmit and receive pair and did NOT see any mention of that. As best I can tell you can shiftout data and pulse it in.

Contact them and see if they have an example project for the arduino. :slight_smile:

Thanks for replying.

However, my interest for the algorithm/lib is not grounded in a need to interface a certain sensor from Sparkfun, but in having the lib for many other purposes as well.

Actually, that's why I created this thread with the subject "Manchester encoding algorithm and library" and didn't reply to the other one which concentrates on a specific use case. :slight_smile:

Hello all,
I've finally got around to trying to use these and would like to have a standalone arduino capable of transmitting data over these RF links.

Currently, for my purposes I only need to be able to send a recognizable "on" or "off" signal to indicate the status of a remote sensor. This will still require some sort of "encoding" so the receiving end can discriminate between static and the intended signal.

So far I have hooked up my 2400bps transmitter to an arduino, and am having the arduino repeatedly send a byte using manchester encoding.

The receiver is just hooked to an oscilloscope sat this time as I wanted to ensure timings were right, etc.

Things seem to be working in order, although I've found erratic results if the transmitter's data pin is not set to "low" after sending the bits.

I do not have any experience with interrupts and for simplicity's sake would rather have the receiving arduino check the airwaves periodically for a transmission. This may not be good for sending actual data, and it will use up a decent chunk of processor time in order to be reliable, but for my purposes will be fine.

I'll post my progress in this thread as things develop.

So far, the test code snips look like (some non-relevant code snipped and ellipsized):

byte xmit;
byte xmitdly = 208;
...
void setup()
{
  pinMode(7, OUTPUT);
...

void loop()
{
...
  digitalWrite(7, LOW);
  delayMicroseconds(xmitdly);
  byte bitv = 128;
  for (int bitc = 0; bitc < 8; bitc++) {
    if (xmit & bitv) {
      digitalWrite(7, HIGH);
      delayMicroseconds(xmitdly);
      digitalWrite(7, LOW);
      delayMicroseconds(xmitdly);
    } else {
      digitalWrite(7, LOW);
      delayMicroseconds(xmitdly);
      digitalWrite(7, HIGH);
      delayMicroseconds(xmitdly);
    }
    bitv = bitv >> 1;
  }
  digitalWrite(7, LOW);

The variables xmit and xmitdly may be set by the computer by code that is out of the context of this discussion, but essentially xmit holds the byte value to transmit.

OK, I have completed a small function which works for my purposes. It works as follows:

The sending arduino now uses a subroutine to send a byte. It looks like this:

void xmitByte(byte vData) {
  byte bitv = 128;
  for (int bitc = 0; bitc < 8; bitc++) {
    if (vData & bitv) {
      digitalWrite(7, HIGH);
      delayMicroseconds(208);
      digitalWrite(7, LOW);
      delayMicroseconds(208);
    } 
    else {
      digitalWrite(7, LOW);
      delayMicroseconds(208);
      digitalWrite(7, HIGH);
      delayMicroseconds(208);
    }
    bitv = bitv >> 1;
  }
  digitalWrite(7, LOW);
}

Note that these timings are hard-coded and go at roughly 2400bps.

For making it easier on the receiving end I chose to begin a transmission with the bytes with ASCII values 0, 0, then 150. This was chosen since it is fairly recognizable and will help the receiving end differentiate between data and static.

I have tested this sending four bytes at a time, repeatedly in bursts: 0 0 150 [any byte] 0

It comes in on the other arduino seemingly reliably. The receiving arduino uses another subroutine which is periodically called by the loop(), and will spend a maximum of 500 milliseconds waiting for legible data to come in. It waits until two of those three bytes (an integer of 150) are seen, then reads the next byte and returns it to the calling function.

byte recvByte() {
  // check for bit change
  unsigned long starttime = millis();
  byte bitin = digitalRead(12);
  byte bitin2 = bitin;
  while (bitin2 == bitin) {
    bitin2 = digitalRead(12);
  }
  bitin = bitin2;
  delayMicroseconds(120);
  //read potential bits
  unsigned int curdata;
  byte lastbit = 0;
  long bitcnt = 0;
  byte bittmo = 0;
  while ((curdata != 150) && (millis() - starttime < 500)) {
    while (bitin2 == bitin) {
      bitin2 = digitalRead(12);
      bitcnt++;
    }
    bitin = bitin2;
    delayMicroseconds(120);
    curdata = curdata << 1;
    if (bitcnt < 47) {
      if (lastbit > 0) curdata++;
      while (bitin2 == bitin) {
        bitin2 = digitalRead(12);
      }
      bitin = bitin2;
      delayMicroseconds(120);
    } 
    else {
      lastbit = 1 - lastbit;
      curdata = curdata + lastbit;
    }
    bitcnt = 0;
    bittmo++;
    if (bittmo > 32) {
      bittmo = 0;
      lastbit = 1 - lastbit;
    }
  }
  if (curdata == 150) {
    byte curbyte;
    byte bitnum = 0;
    while (bitnum < 8) {
      while (bitin2 == bitin) {
        bitin2 = digitalRead(12);
        bitcnt++;
      }
      bitin = bitin2;
      delayMicroseconds(120);
      curbyte = curbyte << 1;
      if (bitcnt < 47) {
        if (lastbit > 0) curbyte++;
        while (bitin2 == bitin) {
          bitin2 = digitalRead(12);
        }
        bitin = bitin2;
        delayMicroseconds(120);
      } 
      else {
        lastbit = 1 - lastbit;
        curbyte = curbyte + lastbit;
      }
      bitnum++;
      bitcnt = 0;
    }
    return(curbyte);
  }
}

void setup() {
  pinMode(13, OUTPUT);
  pinMode(12, INPUT);
  pinMode(11, OUTPUT);
  digitalWrite(13, HIGH);
  Serial.begin(115200);
}

void loop() {
  //  delay(100);
  byte tmp = recvByte();
  if (tmp > 0) {
    Serial.println(tmp, DEC);
  }
  Serial.println(analogRead(0), DEC);
}

Please forgive me for the messy code, it is nothing elegant and probably uses a less common approach to making sense of the bits. To sum it up, the routine looks for the state of the input pin, and checks how long it has taken to change. A longer gap and the current bit is opposite the last, and a shorter delay means the current bit is same as last. Here also the timing is hard-coded, and there are areas of code which could be optimized but I was having trouble doing this (getting erratic results).

In conclusion, this has been my quick hack so I can use these wireless TX/RX in a small project of mine. The code could certainly be cleaned up and possibly put into a library but I certainly don't have time. Hopefully this code will help someone else in the process :slight_smile:

After toying with my small project integrating these RF links, I have found it to be unstable at moderate distances. To improve the reliability I have rewritten the code in order to have it recover the clock. I'm working on cleaning this up and testing it, and once I'm finished I'll post updated code.

@yerg2k: I really appreciate your work on this, and that you made the choice to share it with the rest of us. I just received two pairs of these devices (specifically this one) from SparkFun. I'm more than a little upset at their lack of performance interfacing directly with the hardware serial interface, especially since there walkthrough says that it will work and even gives sample code (that obviously doesn't work).

I'm interested where your development is currently. As I said, I have two of these pairs and I intend to make use of them, even if it means more work than I had initially anticipated. That being said, your code works for me! :slight_smile: I was thinking of cleaning it up, making it slightly more robust and universal, and then packaging it into a class that hopefully more people can use in the future.

What do you think?

anaidioschrono, that is great! I can't seem to get this stuff working because my receiver is having inconsistencies.

I think it is great that you would like to clean it up and put it into a library. I'm usually off to the next project when something like this is worked out, but I'd like to see something like this in a library and it's great to see this made available to those in need of these things.

I ended up changing the receive code twice but have run into an issue. When receiving a stream of 0's the receive data line does not match the transmit. If the transmit is going on/off for ~208 microseconds, the output of the receiver follows the same frequency but at a different duty cycle, being "low" about twice as long as it is "high".

I also changed the receive code to shift bits into an unsigned int, so the "preamble" can be detected in curdata >> 16 and the "packet" in curdata & 0xFFFF. Additionally, some code to detect the "clock" was added. What this is doing is similar to before in detecting how long it takes to go from low to high or high to low. The threshhold time between a long pause (current bit != last) and a short pause (current bit == last) is figured out by the shortest pause, + 1/2 that value.

I'll have to switch back to this old code to see what the difference is between the ways of interpreting, however the newer code may be better for adapting to different baud rates, plus it may be smaller code for doing all the shifting in one loop. Try it out-

// Copyleft 2008 by Derek Yerger

int dbg[2] = { 180, 90 } // timings which can be changed during runtime
  int waitOnlvl(byte vLvl) {  // wait for pin 7 to become vLvl, then wait dbg[1]*2 microseconds
  byte wlvl = digitalRead(7);
  byte wlvl2 = wlvl;
  int lvlcnt;
  while (wlvl2 != vLvl) {
    lvlcnt = 0;
    while (wlvl2 == wlvl) {
      wlvl2 = digitalRead(7);
      lvlcnt++;
    }
    wlvl = wlvl2;
    delayMicroseconds(dbg[1]*2);
  }
  return(lvlcnt);
}

unsigned int recvInt(void) {
  // check for level change
  unsigned long starttime = millis();
  int tmocyc = 0;
  byte reftm = amlvl + 20;
  byte reftm2 = reftm + (reftm / 2);
  //read potential bits
  unsigned long curdata = 0;
  byte lastbit = 1;
  byte lastbit2 = 0;
  byte lastlvl = 1;
  byte lvlq = 0;
  byte cbit = 0;
  waitOnlvl(0);
  waitOnlvl(1);
  while (((curdata >> 16) != 150) && (tmocyc < 5000)) {
    curdata = curdata << 1;
    lvlq++;
    while (lvlq > 0) {
      lastlvl = 1 - lastlvl;
      cbit = waitOnlvl(lastlvl);
      if ((cbit < reftm) && (cbit > amlvl)) {
        reftm = cbit;
        reftm2 = reftm + (reftm / 2);
      }
      lvlq--;
    }
    if (cbit > dbg[0]) {
      lastbit2 = lastbit;
      lastbit = 1 - lastbit;
      curdata = curdata + lastbit;
    } 
    else {
      curdata = curdata + lastbit;
      if (lastbit2 != lastbit) lvlq++;
      lastbit2 = lastbit;
    }
    if ((curdata >> 16) == 65535) {
      curdata == ~curdata;
      lastbit = 1 - lastbit;
      reftm = cbit;
      reftm2 = reftm + (reftm / 2);
      lvlq++;
    }
    tmocyc++;
    if ((tmocyc % 128) == 127) {
      if (millis() - starttime > 1000) tmocyc = 5000;
    }
  }
  if ((curdata >> 16) == 150) {
    return((unsigned int) (curdata & 0xFFFF));
  }
}

I think the main issue will be with how the receive data looks:

And like I said the above (new) code is not working right but it may have to do with the receive data. It may be a good idea for the final library to be more like the above code, though, for reasons stated above, plus having more user data (an unsigned int instead of a byte). Also, some sort of error checking would not be a bad idea. Once I figure out what's the deal with this erroneous(?) duty cycle I'll see what I can come up with for software.

ah! I wish I had a scope... :cry:

All of my work on decoding various protocols has been done using the Arduino. :slight_smile:

I'm plugging away at creating the library. I'm trying to integrate it into SoftwareSerial, I think that makes the most sense. My idea is that the user would get a choice of encoding when instantiating the SoftwareSerial class.

I'm having a bit of a time cleaning the code up, though, as I'm afraid that I don't understand some of it. As an example, how does this work?

if (bitcnt < 47) {

bitcnt just looks to be incremented every time through the loop just above it. Why the number 47? I'm guessing that this is a sort of timer, why not just use one of the real timers?

Oh, I forgot to mention two more things.

  1. Is the choice to send the data out MSB first on purpose, or was this just because there was a 50/50 choice? Most all other communication is LSB first.

  2. It might make locking onto the signal easier if we did something similar to how Ethernet does it. The preamble it a bit pattern that produces a square wave. This makes it really easy to lock as the only times the signal flips is on the center of each bit boundary.

Let me know what you think. I've been pretty busy lately, but I'm going to try and get some time in the next couple of days to sit down and get all this figured out. I'll post some code here as soon as I have something usable.

I've just ordered one of these links and I'll be back home in Australia in under a week so I'll be able to join this effort. :slight_smile:

Btw anaidioschrono, most other things dont use LSB. :stuck_out_tongue:
Your standard computer does but many other things dont.
Also in communications its standard to use network byte order which is MSB.

There are some optimal preambles lying around which allow you to sync very easily with minimum bits sent.

Network byte order is indeed most significant byte (as is most computer to computer communication, except wifi). However, in all networking standards that I am familiar with the bits are transmitted least significant bit first. What protocols are you familiar with that transmit the most significant bit in a byte first?

What protocols are you familiar with that transmit the most significant bit in a byte first?

Many binary protocols use network byte order especially if they are designed to work on multiple platforms.
Some games (OpenTTD for example) and P2P protocols.
Gnutella uses it primarily but it also uses LSB a bit.

Dont microcontrollers use MSB as well?
I seem to remember reading that somewhere.

Update:

Yes, I've had a preamble for locking onto, it is simply 4 bytes of 0's, then the char 150. Also, if the current data buffer >> 16 == 65535 then all the bits are inverted and the processing continues.

The issue with timnings has to do with the bit slicer, which according to http://www.rfm.com/corp/appdata/AN43.pdf

Comparison of the simulation results shows that for a low level signal, the data out of the
comparator, for the 1's, is approximately 30% narrower than the data in. In the case of a
high level signal, the data out of the comparator, for the 1's, is approximately 25% wider
than the data in. This is important to understand if maintaining the data width or pulse
width is critical. When maintaining data width or pulse width is important to the
performance of the receiver system, it will be necessary to either lower the data rate or
increase the low pass filter bandwidth.

So, I'll be working out a workaround for this.

With bitcnt < 47, that number is an arbritrary number paired with the send code and is used to differentiate a long pause or slow pause between transitions. In the latest code I posted, this number is calculated by (shortpause + (shortpause / 2)) where shortpause is the shortest interval it has been between a level change. Obviously this isn't working due to the inconsistencies of the bit slicer output.

Finally! I've figured out what's going on here.

First off, from Laipac TLP/RLP 315mhz - SparkFun Electronics Forum

The HT12E encoder chip used in Laipac's datasheet uses a different type of encoding. For Zeros it outputs "001" and for Ones it outputs "011". With this encoding (after you are syncronized), your receiver will measure how long the signal is high to determine what bit was sent.

This would explain why the "high" not as long as "low" when transmitting a square wave.

So, some modifications were made to the transmitter code to only use the time "high" for meaningful data. This means the data rate will be variable depending on how many 1s.

int dlyms = 208;
void xmitByte(byte vData) {
  byte bitv = 128;
  for (int bitc = 0; bitc < 8; bitc++) {
    if (vData & bitv) {
      digitalWrite(xmitpin, HIGH);
      delayMicroseconds(dlyms*2);
      digitalWrite(xmitpin, LOW);
      delayMicroseconds(dlyms);
    } 
    else {
      digitalWrite(xmitpin, HIGH);
      delayMicroseconds(dlyms);
      digitalWrite(xmitpin, LOW);
      delayMicroseconds(dlyms);
    }
    bitv = bitv >> 1;
  }
}

So a zero is high 208us low 208us, and a one is high 416us low 208us.

Example usage with corresponding preamble:

  for (int xbr = 0; xbr < 3; xbr++) xmitByte(0);
  xmitByte(150);
  xmitByte(curInt >> 8);
  xmitByte(curInt & 0xFF);
  xmitByte(~curInt >> 8);
  xmitByte(~curInt & 0xFF);
  xmitByte(0);

The preamble is: 0x00 0x00 0x00 0x96. The 96 in binary is 01101001. As you can see the transmission includes the MSByte & LSByte of the data followed by its inverse. This will be used for error checking later.

On the receiving end:

// copyleft 2008 by Derek Yerger
byte amlvl = 15; // minimum timeout the routine will use in "clock recovery"
int waitOnlvl(byte vLvl) {
  byte wlvl = digitalRead(7);
  byte wlvl2 = wlvl;
  int lvlcnt;
  while (wlvl2 != vLvl) {
    lvlcnt = 0;
    while (wlvl2 == wlvl) {
      wlvl2 = digitalRead(7);
      lvlcnt++;
    }
    wlvl = wlvl2;
    delayMicroseconds(dbg[1]*2);
  }
  return(lvlcnt);
}

unsigned int recvInt(void) {
  // check for level change
  unsigned long starttime = millis();
  int tmocyc = 0;
  digitalWrite(ledPin, HIGH);
  byte reftm = amlvl + 20;
  byte reftm2 = reftm + (reftm / 2);
  //read potential bits
  unsigned long curdata = 0;
  unsigned int preamble = 0;
  byte lvlq = 0;
  byte cbit = 0;
  byte abc = 0;
  waitOnlvl(0);
  waitOnlvl(1);
  while ((preamble != 150) && (tmocyc < 5000)) {
    preamble = (preamble << 1) + (curdata >> 31);
    curdata = curdata << 1;
    cbit = waitOnlvl(0);
    if ((cbit < reftm) && (cbit > amlvl)) {
      reftm = cbit;
      reftm2 = reftm + (reftm / 2);
    }
    if (cbit > reftm2) curdata++;
    waitOnlvl(1);
    tmocyc++;
    if ((tmocyc % 128) == 127) {
      if (millis() - starttime > 1000) tmocyc = 5000;
    }
  }
  if ((preamble == 150) && ((curdata >> 16) == ~(curdata & 0xFFFF))) {
    digitalWrite(ledPin, LOW);
    return(curdata & 0xFFFF);
  }
}

Here the routine shifts data in, shifting the MSB of curdata into preamble. When preamble is 150 the loop exits. This means two bytes (0x00, 0x96) need to shift into preamble before the loop exits, or until 1000 ms has elapsed. The tmocyc var is to avoid calling millis() too often.

Once the loop exits, the preamble is checked, and the data in curdata compared for the integer and its inverse. If this matches up the function returns the number.

I've tested this with a counter and it is working great. The proper data comes out, and no erroneous data is returned either. I hope this helps all of you in the same situation I was a week ago. Please keep this thread updated with any libraries that come of this. Also, I'm sure there are minor improvements which could be made to make the code smaller.

just to be clear... your all using encoding that is done on the arduino, so no encoder hardware is needed... right?

That is correct.

yay! no money to spend!

:slight_smile:

OK, now I'm really confused.

Although this code is working 100% all good, I've found something that just does Not Make Sense. Check it out:

On the sending end the data goes out in this order:

  xmitByte(curInt >> 8);
  xmitByte(curInt & 0xFF);
  xmitByte(~curInt >> 8);
  xmitByte(~curInt & 0xFF);

So, first the proper data is sent, then its inverse goes out last.

On the receiving end, data is shifted in one bit at a time into curdata, so the last bit in is LSB and the least significant two bytes should be the current int's inverse, right? Well here's the code which is working:

if ((preamble == 150) && ((curdata >> 16) == ~(curdata & 0xFFFF))) {
    digitalWrite(ledPin, LOW);
    return(curdata & 0xFFFF);
  }

So the function returns curdata & 0xFFFF which technically should be the current data inverse. But everything works this way, and the number my function returns correlates with what the transmitter is sending. And it isn't that all of the bits are flipped, because the preamble is still properly recognized.

But when I change the return to what it "should" be (curdata >> 16) then the code stops working and only returns one of two numbers (00 00 or 8D A7). This makes no sense, am I just missing something blatantly obvious here?

Of course, I'm happy to have my RF links up and running as they should be, but I still am curious about why the code behaves like this. Can anyone point me in the right direction?

I really don't know how it works like this. If curdata == 0xAAAA5555 and you put it through

((curdata >> 16) == ~(curdata & 0xFFFF))

then (curdata >> 16) = 0x0000AAAA
and ~(curdata & 0xFFFF) = 0xFFFFAAAA

this is because curdata & 0xFFFF evaluates first giving 0x00005555, then the ~ is evaluated giving 0xFFFFAAAA.

I don't understand how that expression can ever evaluate to true. :frowning: If somebody knows that part of the puzzle please let me know! [smiley=huh.gif]

you might want to try something like this:

(((curdata >> 16) & 0xFFFF) == (~curdata & 0xFFFF))