Decoding RF signal from Ambient Weather Thermo-Hygrometer

[3zero8]

For my first RF project, I'm attempting to capture and decode 433 MHz RF signals received from a remote Thermo-Hygrometer sensor in order to perform some home automation at various temperature and humidity levels.  This is the Thermo-Hygrometer that I am using:
http://www.ambientweather.com/amws08x2kit.html

I'm able to capture the RF signals in Audacity using a setup similar to this one:
http://davehouston.net/learn.htm

I have attempted to decode the signals with NRZ and Manchester coding then searching the results for the BCD and binary representation of the decimal temperature/humidity, in various bit and byte endianness, but have not been successful.

I would like to know:
> Is there anything I am missing?
> Am I using the correct decoding?
> Any suggestions or next steps?

Signal Screen Captures:

18.6 C 36%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF2.png

000000000011100101001100000100000000101000101011000000000000011100101001100000100000000101000101011000000000000011100101001100000100000000101000101011000

18.6 C 36%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF3.png

000000000011100101001100000100000000101000101011000000000000011100101001100000100000000101000101011000000000000011100101001100000100000000101000101011000

18.7 C 44%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF6.png

000000000011100101001100000100010000001010000011000000000000001110010100110000010001000000101000001100000001000000111001110011000001000100000011100111110000

19.2 C 43%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF7.png

000000000011100101001100000100011100110000010100000000000000111001010011000001000111001100000101000000000000001110010100110000010001110011000001010000

19.3 C 38%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF8.png

000000000011100101001100000100010100010000100011000000000000011100101001100000100010100010000100011000000000000011100101001100000100010100010000100011000

Signal pattern repeated 3 times:
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/Pattern.PNG

RF Signal Width Distribution Graph:
I used this to determine what should be considered a long pulse and a short pulse
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/Signal_Width_Distribution.png

Thank you in advance for any assistance you can provide!

[jremington]

I couldn't make out the signal traces on my screen, but with long and short pulse you are on the right track.

Temporarily assign 1 and 0 to one or the other and compare the patterns you see with the numbers you know are being transmitted. There is probably a preamble, a device ID, some data coded as either binary or binary coded decimal, and a checksum of some sort.

There are lots of examples on the web of similar transmissions that have been successfully decoded. Here is a collection at http://fredboboss.free.fr/tx29/index.php

[3zero8]

Thank you for the encouragement jremington; I feel a little better having someone tell me that I'm on the right track.

I have performed a great deal of research, testing, sampling, and analyzing before posting here.  As you noted there are a lot of great resources for decoding weather station signals and RF signals in general.  I will share a few that I have come across in case they are useful to someone else:

http://forum.arduino.cc/index.php/topic,110662.msg840578.html#msg840578
http://forum.arduino.cc/index.php/topic,112699.0.html
http://forum.arduino.cc/index.php?topic=145341.0
http://lucsmall.com/2012/04/29/weather-station-hacking-part-2/
http://www.f6fbb.org/domo/sensors/tx3_th.php
http://thekidd.ca/blog/tech/home-automation-ac-wall-plugs-rf-modules-and-an-arduino
http://rayshobby.net/?p=3381
http://bertrik.sikken.nl/433mhz/

As for the message, I believe that the entire data stream is repeated three times within each transmission.  You can see the repetition in the following:



When comparing multiple RF transmissions, one can see that the first 32 bits, in red below, never change… this I assume is the preamble, device id, unused data fields, and possibly a check sum.

(Each new line below is a separate transmission)
0000000000111001010011000001000000010010100001010000
000000000011100101001100000100000000101000101011000
000000000011100101001100000100000000101000101011000
0000000000111001010011000001000000010010001010010000
0000000000111001010011000001000000010010001010010000
00000000001110010100110000010001110011000001010000
000000000011100101001100000100010100010000100011000
0000000000111001010011000001000110101001001011000
0000000000111001010011000001000111001011001010000
00000000001110010100110000010001101010100100001000

If you remove the static bits, the bits below remain to store the temperature and humidity…

(Each new line below is a separate transmission)
10010100001010000
101000101011000
101000101011000
10010001010010000
10010001010010000
1110011000001010000
10100010000100011000
110101001001011000
111001011001010000
1101010100100001000

According to the manufacturer, the humidity is in the range of 10-99 which I believe would require 7-bits minimum to store.  The temperature values are from -40 to 140 degrees F, again I believe requiring a minimum of 7-bits for a total of 14-bits to transmit both humidity and temperature.

None of the signals are shorter than 14-bits so this is feasible… but no matter how I flip the bits or the order I read them I am not able to decode... This is what causes me to suspect that I may be interpreting the signal incorrectly...

[jremington]

Decoding that signal may not be easy, but it is an interesting challenge.

It is very helpful to compare signals that differ by the minimum possible number of bits. For example, if you could compare several signals reporting exactly the same humidity, but minimal variations in temperature of for example 18.5, 18.6, 18.7 etc. degrees, that will help a lot. Also, if you can capture a temperature of exactly 0.0 degrees, that can be very useful.

Another useful piece of information is how a presumed checksum (usually one byte, but I've seen a 4-bit checksum) varies with any change in the message. If the message changes by exactly one bit, but many checksum bits change, then it is probably a CRC type of checksum. If only one checksum bit changes when one message bit changes, it is probably a simple sum.

[robwlakes]

Hi folks I have "hacked" two different brands of weather stations, the Bios (or Thermor) and the Oregon Scientific V3.0 and been able to intercept the signals successfully.  I have documented this in these two postings where source code (plus comments!) are provided -
http://forum.arduino.cc/index.php?topic=167364.msg1432791#msg1432791 The Bios weather station
http://forum.arduino.cc/index.php?topic=203425.0 Oregon Weather  Station V3.0 protocols

I had to ditch the Bios as it just wore out, and I am now concentrating on the Oregon system. The Oregon is now providing local weather information for my website http://www.laketyersbeach.net.au/

Both systems use Manchester encoding, but with opposite polarity conventions.  The signal frame's timing was similar but the data structures were quite different.

The Bios had a single transmitter (all the sensors were wired to a common point), no checksum and varying length data payloads, but transmitted 4 copies of the data for three different sensors every 40secs (approx). So it took 120 secs to get all the data.  The repetition meant receiving 4 identical data payloads = data is ok.

The Oregon has separate transmitters in the rain gauge, anemometer and temperature, and all periodically transmit a single data payload in different length cycles (eg anemometer say every 20 seconds, rainfall every 80 seconds) , so some overlapping data corruption is inevitable.  However it has a constant length data payload, and employs a simple checksum routine to check the integrity of the data.  The Oregon is versatile in that other sensors from the V3.0 format could be added easily to the line up eg a UV light detector.

Both designs used a rolling ID number that changed every time it was reset or batteries were changed and both used a mixture of BCD and pure binary in number formats.  Both used a stream of ones to synch the receiver, about 80 for the Bios and about 20 for the Oregon.  The Bios had its first zero in the data stream outside of the data byte boundaries, whereas the Oregon has the first zero within the data byte boundaries and hence part of the checksum.

You may find the two programs interesting in your quest to understand the protocol even though they are different brands to yours.  I employ a system of delays to sample the waveform rather than counting with interrupts and then decode the data.

I will look at the progress with interest, Cheers, Rob

[robwlakes]

Sorry to not have spotted this earlier 3zero8, but I missed your very long and thin graphics of your signal sampling.  I have had a much closer look at your data.

I noticed that you were assuming the data stream begins with 0's when most Manchester formats begin with 1's.  This made me suspect that analysis of the data stream into 0&1's may not be correct.

I have only worked on the first 20 or so bits as a Manchester encoding to test the water and see what you think.  I think it is encoded with a 0->1 signal transition in the middle of the bit period means a 1, and a 1->0 signal transition is a zero (so similar to the Bios).  The black arrows going down, are where zeros are found and where the green arrows are going upwards are ones. (The signal transitions not so marked are actually at the start and finish of the of the full bit patter. The signal transitions that indicate the data, 0 or 1 are in the middle of this, and when the data stream swaps from 0->1 or a 1->0 there is actually no transition at the end and start of the bit patterns.  Weird but true!  It allows the signal polarity to swap.).  If the green and black arrows are evenly spaced that makes it likely it is Manchester.

As you can see your bit stream of 0&1's is quite different from mine.  I maybe wrong with this attempt but I think it is worth considering.
Yours- 000111001010
Mine- 11110101000101100
So if it is Manchester then discard the first 4 '1's' above as a header of ones to stabilise the AGC on the RX, so no data in them.
The next 0 is what synch's the rest of the data stream and may or may not be included in the data byte boundaries.
The rising edges in the header zone is what the search program synch's with and sets the place where all other transitions are sampled.  Signal 0->1=logic 1 and Signal 1->0= logic 0. The first time a 1->0 transition occurs at that time, and this then marks where data payload begins.
Keeping your program locked into this waveform format is the next challenge, and my programs I have listed in my previous post will help you with that sampling.
When you can confidently get a stable stream of  a header, the start 0, and data payload as a binary list then you can more confidently start picking the patterns to unravel the weather data hidden within.  Using a dump routine that formats the binary better than Serial.print(nos, BIN) which strips leading zeroes will help a lot in getting the byte boundaries worked out (again there is an example in my programs or see below).  Also be aware that if they use BCD encoding of numbers, then they will not conform to the "such and such number range" will require so many bits to be encoded.  Decoding the bytes to Hex can help detect BCD.

Code: [Select]


void binBank(){
 //Print the fully aligned binary data in manchester[] array
 Serial.println("D 00 00001111 01 22223333 02 44445555 03 66667777 04 88889999 05 AAAABBBB 06 CCCCDDDD 07 EEEEFFFF 08 00001111 90 22223333");
 Serial.print("D ");
 for( int i=0; i < maxBytes; i++){
   byte mask = B10000000;
   if (manchester[i]<16){
     Serial.print("0"); //pad single digit hex
   }
   Serial.print(manchester[i],HEX);
   Serial.print(" ");
   for (int k=0; k<8; k++){
     if (manchester[i] & mask){
       Serial.print("1");
     }
     else{
       Serial.print("0");
     }
     mask = mask >> 1;
   }
   Serial.print(" ");
 }
 Serial.println();
}


I hope I am on the right track, and this will help you.
Let me know what you think.

Rob

 

[jremington]

If this signal is Manchester encoded, then the encoding appears to violate the premise that the transition occurs in the middle of the bit period. The spacings between the green and black arrows in robwlakes' proposal aren't equal. However, at least one of the three Oregon Scientific protocols is a Manchester code with an off-center transition, so that is certainly possible.

I agree that transmissions usually start with a collection of "1" bits, so it would be useful to reverse the 1/0 assignments you've already made to see if that helps. It would also be good to measure the LOW pulse widths to see how their statistics look. If the signal is Manchester encoded, there should be two pulse widths for LOW and two for HIGH, with approximately equal timings. If that turns out to be the case the decoding will be easy.

Post your results and we will try to help!  Again, if you can, collect transmissions that differ by the minimum possible number of bits.

Edit: After looking at one of the transmissions a bit more closely, I tend to agree with robwlakes assessment that the encoding is Manchester, but the bit transition does not occur in the middle of the interval. A bit time of about 48 pixels seems to fit the data for RF2.png. A transition late in the interval, from low to high, is likely to be a "1" and a transition from high to low, early in the interval, is likely to be a "0" as he suggested.

[3zero8]

Hello everyone and thank you for your assistance.  I had a chance to collect additional samples and work on this problem some over the weekend. 

JRemington I took your suggestion to eliminate the humidity as a changing variable in the data.  To ensure a consistent humidity reading, I cracked open the sending unit and hacked the humidity sensor.  Also per your recommendation, I collected a number of temperature readings which varied by only .1 degree C.  I was able to do so by placing the sensor in close proximity to a light bulb which steadily increased the temperature without any jumps.

Rob I had previously come across your BIOS and Oregon Scientific station posts and they are a wealth of information; I studied the Oregon Scientific protocols document that you posted at length a couple weeks ago.  As far as the Ambient Weather signal, you were spot on with the Manchester encoding. I had decoded a few signals by hand this way, but after not finding the values I was looking for, had given up thinking that the signal widths deviated too greatly for Manchester encoding.

With your helpful posts, I believe that I have successfully extracted temperatures and humidity from a number of samples, however analysis on a wider range of data is needed to confirm.  I will also be attempting to determine the checksum area and method, which I believe is in the last 2 bytes of data.

[3zero8]

Samples with Temperature and Humidity positions

																																			T	T	T	T	T	T	T	T	T	T	T	H	H	H	H	H	H	H	H
21.1C 11RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	0	1	0	1	1	0	0	0	0	1	0	1	1	0	1	0	1	0	0	1	0	0	0	0	0
21.0C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	0	1	0	1	0	0	0	0	0	1	1	0	1	0	1	1	1	0	0	0	1	0	0	0	0
21.1C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	0	1	0	1	1	0	0	0	0	1	1	0	1	0	0	1	1	0	0	1	0	0	0	0	0
21.2C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	0	1	1	1	0	0	0	0	0	1	1	0	1	0	1	0	0	1	1	0	0	0	0	0	0
21.3C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	0	1	1	1	1	0	0	0	0	1	1	0	1	0	0	0	0	1	1	1	1	0	0	0	0
21.4C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	1	0	0	1	0	0	0	0	0	1	1	0	1	1	1	1	1	1	1	1	1	0	0	0	0
21.5C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	1	0	0	1	1	0	0	0	0	1	1	0	1	1	0	1	1	1	1	0	0	0	0	0	0
21.6C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	1	0	1	0	0	0	0	0	0	1	1	0	1	0	1	0	0	0	1	0	0	0	0	0	0
21.6C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	1	0	1	0	0	0	0	0	0	1	1	0	1	0	1	0	0	0	1	0	0	0	0	0	0
21.7C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	1	0	1	1	1	0	0	0	0	1	1	0	1	1	0	0	0	0	0	0	1	0	0	0	0
21.8C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	0	1	1	0	0	0	0	0	0	0	1	1	0	1	0	0	0	0	0	0	1	1	0	0	0	0
22.2C 13RH	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	0	0	0	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	0	0	1	1	0	1	0	1	0	1	0	1	0	0	0	0	0	0

Given the following example: 21.3C 13RH 11111111111010100010101001111000001000100111100001101000011110000

Extract Temperature:
Temperature bits: 10001001111
Convert to decimal: 1103
Equation: (1103 - 400) * 0.1 = 70.3 or 21.3C (rounded)

Extract Humidity:
Binary: 00001101
Convert to decimal: 13 RH

[jremington]

Good job!

I'm guessing that the next 8 bits after the humidity is a CRC-type checksum. If you convert the message to hex, then reveng is a program to help work out the checksum polynomial. The source code for reveng is here http://reveng.sourceforge.net/  See this thread for an example of reverse engineering the CRC calculation of an Oregon Scientific weather sensor: http://sourceforge.net/p/wmrx00/discussion/855158/thread/62e765df

I used Code::Blocks to make a PC console application of reveng.

[robwlakes]

Hi 3zero8,
More on your checksum/CRC challenge -
http://www.youtube.com/watch?v=UAtmQKvMuoQ
suggests it is

Checksum is CRC8 (polynomial x8+x5+x4+1 = 100110001)

And if that does appear to be the case, then this may help set it up http://www.leonardomiliani.com/2013/un-semplice-crc8-per-arduino/?lang=en

The V3.0 Oregon Scientific was a simple 4-bit sum of the preceding Nibbles, much simpler than a CRC8 polynomial. Plus if the same payload is transmitted identically more than once in a packet then it may not have a checksum of any sort (as per the Bios).

Nice work on the decoding so far, excellent progress,

Rob

[robwlakes]

How's it going 3zero8?
I don't want to put any pressure on, but I am intrigued as to how you are progressing.  I was very tempted to buy the Ambient system myself, and would love to know what the final format was.  The integrated solar system was very attractive as well as the overall integrated package on the one "pole". Plus suppliers also seemed to sell spares on the internet as well.  I can't change brands myself now, but your work will be appreciated by many I am sure. Will you go on to do the Anemometer and rainfall etc as well?
Cheers, Rob

[jenfour6]

For my first RF project, I'm attempting to capture and decode 433 MHz RF signals received from a remote Thermo-Hygrometer sensor in order to perform some home automation at various temperature and humidity levels.  This is the Thermo-Hygrometer that I am using:
http://www.ambientweather.com/amws08x2kit.html

I'm able to capture the RF signals in Audacity using a setup similar to this one:
http://davehouston.net/learn.htm

I have attempted to decode the signals with NRZ and Manchester coding then searching the results for the BCD and binary representation of the decimal temperature/humidity, in various bit and byte endianness, but have not been successful.

I would like to know:
> Is there anything I am missing?
> Am I using the correct decoding?
> Any suggestions or next steps?

Signal Screen Captures:

18.6 C 36%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF2.png

000000000011100101001100000100000000101000101011000000000000011100101001100000100000000101000101011000000000000011100101001100000100000000101000101011000

18.6 C 36%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF3.png

000000000011100101001100000100000000101000101011000000000000011100101001100000100000000101000101011000000000000011100101001100000100000000101000101011000

18.7 C 44%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF6.png

000000000011100101001100000100010000001010000011000000000000001110010100110000010001000000101000001100000001000000111001110011000001000100000011100111110000

19.2 C 43%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF7.png

000000000011100101001100000100011100110000010100000000000000111001010011000001000111001100000101000000000000001110010100110000010001110011000001010000

19.3 C 38%RH
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/RF8.png

000000000011100101001100000100010100010000100011000000000000011100101001100000100010100010000100011000000000000011100101001100000100010100010000100011000

Signal pattern repeated 3 times:
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/Pattern.PNG

RF Signal Width Distribution Graph:
I used this to determine what should be considered a long pulse and a short pulse
https://dl.dropboxusercontent.com/u/54300346/WS-08RF/Signal_Width_Distribution.png

Thank you in advance for any assistance you can provide!

Sounds like a good plan, just on a side note remember to regularly re-calibrate the thermo-hygrometer.

[robwlakes]

hi jenfour6,

I have looked at the first captured waveform image in your list and I believe you are dealing with Manchester encoding.  I have gone through and annotated it below the waveform with what I reckon are the correct bit decoding (see attachment).  There are three packets I believe, as the bit pattern repeats three times ABC.

If you would like a good intro to Manchester encoding I have made an effort to explain it here https://github.com/robwlakes/ArduinoWeatherOS.  Briefly it is based on transitions rather than durations (each bit, 0 or 1 has the same amount of "lo and hi" time) and in your case lo->hi =1 and hi->lo=0. The trick is being able to recognise what is happening in the middle of any bit pattern.  Transitions in the middle tell what the data bit is.  Then being able to ignore the transitions (or no transition!) that occur at the start or end of a bit pattern, as they don't matter.  My article is about my Oregon Scientific WS which has a Manchester polarity that is the reverse hi->lo=1 and lo->hi=0, but the general principles of understanding the code and how to decode are provided there.

My older Forum article http://forum.arduino.cc/index.php?topic=167364.0 on Bios/Thermor WS's will also be very relevant as it covers Manchester encoding with same polarity as yours, and it also uses a "triple-packet-send" as a data validation process.  It also has handy debug printing routines that will help you print the packet in hex and then this will allow you to put together the bytes in the packet, and the environmental readings, easier. eg dumping the whole packet to a hexadecimal string.

Cheers, Rob

[Tops]

Hello,

I was pleased to find this forum - I am trying to do the same thing but am stuck. First I thought I should share some of the info I think I have deduced so far which is not yet here. Using the binary from the 21.1C 11RH example already posted-

The first 21 bits appear to always remain constant
111111111110101000101

The next 8 bits appear to reset randomly each time there is a battery change
01001111

The last three bits of the next 4 appear to be the channel number. There are eight channels that can be set using the dip switches on the back of the unit and these are represented 000 (Channel 1, sent as 0) through to 111 (Channel 8 sent as 7). All the examples shown in this thread appear to be from a unit set to channel one. As the first bit is not required to communicate channel number it may have another purpose.
0 000

The following single bit is in fact part of the temperature reading, it indicates whether the reading is positive (0) or negative (1).
0

The rest is as previously discussed - 11 bits of temperature reading, 8 bits of humidity then 8 bits of CRC.
10001001011 00001011 01010010

Anyway I was wondering if anybody had got a working sketch, I am still having difficulties with
1. Separating the signals from multiple sensors
2. Cracking the CRC
3. Getting my code to work

Andrew