Pages: 1 [2]   Go Down
Author Topic: Decoding RF signal from Ambient Weather Thermo-Hygrometer  (Read 9698 times)
0 Members and 1 Guest are viewing this topic.
Lake Tyers Beach
Offline Offline
Full Member
***
Karma: 2
Posts: 131
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Andrew,

Pleased to be of any assistance.

Quote
The first 21 bits appear to always remain constant
111111111110101000101
  This would probably be the header (1's + a Zero) and the ID of the type of unit it is in the Ambient range.

Quote
The next 8 bits appear to reset randomly each time there is a battery change
01001111
  this is to distinguish your system (hopefully) from any other nearby Ambient transmitter of the same type, eg a Neighbour's WS with the same ID

Your next analysis I can only agree with as having a system to experiment with is the only way of checking, ie check Arduino outputs with what the Ambient console is showing.  I don't have one to experiment with.  However going on your evidence I have doctored my debug version for my old Bios/Thermor WS for you to try and further develop if you get any success from it.

I believe the Bios protocol shares a few important aspects with the Ambient.  Such as lo-hi=>1, hi-lo=>0 for the Manchester Encoding.  I am not sure if it has the first (sync) 0 within the byte boundaries or outside them.  I have indicated in the program where you can alter that to try either way.  You have to look where your other data is falling to work it out.  eg does the temp and humidity data fall squarely within bytes?  The Bios WS did not have a checksum as it sent the whole packet 4 times over each time, so getting 4 identical packets was how the data was validated.  Given the Ambient sends the data 3 times I have adjusted the program to look for 3 identical packets.  Also the Bios had about 80 header bits, more than enough I think!!!  The Ambient looks to have about 10 or 11 so I have adjusted the program to look for 7, so we have a few left over (the program just dumps any excess).

The program is also set up for you to dump the whole three packets into a byte array and then dump this out as binary for your inspection.  At the moment it only dumps out when it gets three identical packets, however you may need to change it to have it dump/debug all three packets separately to look at what you are getting if they are not identical first up.  The program is set to capture 6 bytes (I counted 54 bits in the packet (including a few 0's at the end that looked like padding) so I thought 6x8=48 should be an ok number to collect.  Change it to 5 if you find you can't validate with 6.

The last bits after the humidity could be just junk for padding. Anyway you can have a look as see what they do once you get it dumping binary/hex.  For example all the data you really need to check maybe in the first 5 bytes and you will be able to ignore the last byte if it has random junk in it.  However it could also be a checksum as you say.  My Oregon Scientific WS on the other hand does not repeat the sensors's packet and has a simple summing check-byte at the end of the data bytes, not a difficult polynomial check like some have.  I do have a reference to one of these I can dig out, but I would be betting on no check sum with a repeat of three packets occurring.

The rest of the program is to do with processing the packets according to the Bios WS rules, you will need to devise your own for the Ambient to suit your application, however I hope they don't just get in the way and do help you formulate your own calculation routines.

I have run a compile check on the program and it appears error free in that respect, however the chances I have cracked a perfect score for you so it "runs straight out of the box" are very slim, so I wish you all the best to get any final wrinkles out of it.  Eg enable the printing of the "H" when a valid header is received, if you need to check that).

Let us know how you go, best wishes,

Rob

 

* DebugVersion.ino (18.48 KB - downloaded 19 times.)
Logged

Learning Flute and C++, heading for a meltdown.

Offline Offline
Newbie
*
Karma: 0
Posts: 6
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Rob,

Thank you so much for your excellent articles on decoding weather signals and for your encouragement. I have had lots of fun playing with the code.

I have taken a while to reply as I still have not got anything working and I wanted to check obvious possibilities before reposting.

First - I know that my hardware is set up correctly. I used the code below to monitor the RX input.

Code:
//Define settings
const int INPUT_PIN = 8;                                       //Input pin
const unsigned int MAX_SAMPLE = 256;                           //Maximum number of samples to record
const unsigned long MAX_TIME = 30000UL;                        //Maximum record time after start pulse (msec)

//Variables used in the ISR
volatile boolean running = false;
volatile unsigned long last = 0;
volatile unsigned int count = 0;
volatile unsigned long samples[MAX_SAMPLE];

void setup() {
  pinMode(INPUT_PIN,INPUT);
  Serial.begin(9600);
  while (!Serial) {};
  Serial.println("RecordPin sketch started");
}

void loop() {
  unsigned long startTime;
  int startState = 0;
 
  while (Serial.read() != -1) {};                              //Clear the serial input buffer
  Serial.println("Send character to start");
  while (Serial.read() == -1) {};                              //Wait until we receive the first character
  while (Serial.read() != -1) {};                              //Clear the rest of the buffer
  startState = digitalRead(INPUT_PIN);                         //Save the initial port state
  attachInterrupt(0, receiver, CHANGE);                        //Interrupt 0 = Pin 2
  count = 0;
  last = micros();
  running = true;
  Serial.print("Running. Send a character to stop");
  startTime = millis();
  while (running && ((millis() - startTime) < MAX_TIME)) {     //Run until the buffer is full, the time is up, or we receive a character
    if (Serial.read()!= -1) {                                  //Stop if we receive a character
      running = false;
    }
  }
  Serial.println("Stopped");
  detachInterrupt(0);
  Serial.print("I recorded ");
  Serial.print(count);
  Serial.println(" samples");
  for (unsigned int x = 0; x < count; x++) {
    Serial.print("Pin = ");
    if (startState == 0) {
      Serial.print("Low ");
    } else {
      Serial.print("High");
    }
    startState = !startState;
    Serial.print(" for ");
    Serial.print(samples[x]);
    Serial.println(" usec");
  }
  Serial.println("");
}

//Pin change interrupt routine
//Called when the state of the input pin changes
void receiver()
{
  if (running) {                                               //If we are running
    samples[count] = micros() - last;                          //Save the time from the last change to this one
    last = micros();                                           //Reset the time of last change
    count++;                                                   //Go to next buffer entry
    if (count > MAX_SAMPLE) {                                  //If we are past the last entry
      running = false;                                         // we stop
    }
  }
}

The results look like this

Code:
Pin = High for 480 usec
Pin = Low  for 488 usec
Pin = High for 484 usec
Pin = Low  for 480 usec
Pin = High for 480 usec
Pin = Low  for 484 usec
Pin = High for 492 usec
Pin = Low  for 476 usec
Pin = High for 488 usec
Pin = Low  for 484 usec
Pin = High for 484 usec
Pin = Low  for 492 usec
Pin = High for 472 usec
Pin = Low  for 488 usec
Pin = High for 484 usec
Pin = Low  for 484 usec
Pin = High for 488 usec
Pin = Low  for 484 usec
Pin = High for 476 usec
Pin = Low  for 968 usec
Pin = High for 980 usec
Pin = Low  for 960 usec
Pin = High for 1004 usec
Pin = Low  for 920 usec

To make sense of this I have manually decoded the packet using hi-lo => 1 and lo-hi => 0. From sampling hundreds of highs and lows I also know that the average bit is 968 uSecs. So below is the decoding the results above (I have split the longs up to show the result):

Code:
Pin = High for 480 usec    RX State 1
Pin = Low  for 488 usec    RX State 0 (rising Manchester logic 1)
Pin = High for 484 usec    RX State 1
Pin = Low  for 480 usec   RX State 0 (rising Manchester logic 1)
Pin = High for 480 usec    RX State 1
Pin = Low  for 484 usec    RX State 0 (rising Manchester logic 1)
Pin = High for 492 usec    RX State 1
Pin = Low  for 476 usec   RX State 0 (rising Manchester logic 1)
Pin = High for 488 usec    RX State 1
Pin = Low  for 484 usec    RX State 0 (rising Manchester logic 1)
Pin = High for 484 usec    RX State 1
Pin = Low  for 492 usec    RX State 0 (rising Manchester logic 1)
Pin = High for 472 usec    RX State 1
Pin = Low  for 488 usec   RX State 0 (rising Manchester logic 1)
Pin = High for 484 usec    RX State 1
Pin = Low  for 484 usec    RX State 0 (rising Manchester logic 1)
Pin = High for 488 usec    RX State 1
Pin = Low  for 484 usec   RX State 0 (rising Manchester logic 1)
Pin = High for 476 usec    RX State 1
Pin = Low  for (484) usec  RX State 0 (rising Manchester logic 1)
Pin = Low  for (484) usec   RX State 0
Pin = High for (490) usec   RX State 1 (falling Manchester logic 0)
Pin = High for (490) usec   RX State 1
Pin = Low  for (480)usec   RX State 0 (rising Manchester logic 1)
Pin = Low  for (480)usec   RX State 0
Pin = High for (502)usec    RX State 1  (falling Manchester logic 0)
Pin = High for (502)usec    RX State 1
Pin = Low  for (460) usec    RX State 0 (rising Manchester logic 1)


You may now recognize the sample above as the header with the first two bits of the first byte at the end.

Now to the problem: I cannot capture the data packets! I have tried both the sketch you provided and the interrupt sketches.

Interrupt sketches find the data but it does not match the binary I am decoding manually, the Debugversion sketch reliably finds the header and first 0, but the data packet is very rarely valid (I have captured four in the last week) and the valid captures do not match the binary I extract manually.

I would be very grateful if you, (or any other interested readers) could give me some ideas to correctly capture the four bytes of binary that follows the header and first byte.

Header
1111111111101
First byte (constant)
01000101

Many thanks!
Logged

Lake Tyers Beach
Offline Offline
Full Member
***
Karma: 2
Posts: 131
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Tops,

Well done. You have certainly done some excellent research.  I have been helping another person tracking down their weather station decoding on different thread and over the weekend I have tried to produce a general purpose Manchester decoder that people can use to at least get the raw binary in bytes from their weather stations or what ever.  I would expect that once they have achieved that much they will then be happy go on to decipher the packet code add their own calculations etc

You can find the Debug program here:
https://github.com/robwlakes/ArduinoWeatherOS and it's called DebugManchester.ino.

It would seem that you have got some solid data of 960uS for the bitWaveform duration.  So in the program I have tried to produce (above) try using sDelay=240, and and lDelay=480, and use Polarity= 1, ie negative polarity. ( the first 960approx pause is low, so the rising edge at the end of that is a 0, so lo->hi=0).  I disagree with how you have interpreted the timings.  I think it should decode as  1 1 1 1 1 1 1 0 1 0 1. (My commas separate the bit waveforms, and approx 960 is a full waveform).
Code:
Pin = High for 492 usec  hi
Pin = Low  for 476 usec  lo = = 1,
Pin = High for 488 usec  hi
Pin = Low  for 484 usec lo  ==1,
Pin = High for 484 usec  hi
Pin = Low  for 492 usec  lo ==1,
Pin = High for 472 usec  hi
Pin = Low  for 488 usec lo == 1,
Pin = High for 484 usec  hi
Pin = Low  for 484 usec  lo ==1,
Pin = High for 488 usec  hi
Pin = Low  for 484 usec  lo ==1,
Pin = High for 476 usec   hi
Pin = Low  for 968 usec  lo ==1, lo    
Pin = High for 980 usec   hi ==0, hi
Pin = Low  for 960 usec  lo ==1, lo
Pin = High for 1004 usec hi ==0, hi
Pin = Low  for 920 usec   lo ==1, lo

Hopefully this will make good use of your excellent efforts so far and give you the bytes you are looking for.  I have tried to produce a recipe style program so change those 3 values in the set up, and hopefully you will see results  smiley-roll-sweat  There is also a short guide to how to use the program called Debug.md.

Let me know how you go...

Cheers, Rob  
« Last Edit: July 27, 2014, 05:45:46 pm by robwlakes » Logged

Learning Flute and C++, heading for a meltdown.

Offline Offline
Newbie
*
Karma: 0
Posts: 6
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Thanks Rob,

You are absolutely right with your analysis of the timings - I mucked up the copy pasting when I posted it!

The new sketch works brilliantly and I am now reliably getting all the data - the only problem is aligning the bytes.

The first 0 and the following 1 are both outside the 1st byte - How would I change the program to nudge everything one along when excluding the 1st zero?

Andrew
Logged

Lake Tyers Beach
Offline Offline
Full Member
***
Karma: 2
Posts: 131
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Andrew,
Very good news indeed.  I have altered the DebugManchester.ino program (and renamed it as well, GIT-Hub can track the versions for me) so that the number of leading bits to discard can be chosen.  Limiting it it to just either keeping or dumping the leading zero proved to be too limited as you have shown.

https://github.com/robwlakes/ArduinoWeatherOS DebugManchester.ino

I have just introduced a variable "discards" to set to the number to dump, so in your discards=2.  Whereas in my case I keep them all so discards=0, others would use discards=1 to just dump the leading zero.

I would love to know how you went about figuring out your byte boundaries and what processing was required for your data bytes to turn them into human readable form?

Cheers and keep up the good work,
Rob
Logged

Learning Flute and C++, heading for a meltdown.

Lake Tyers Beach
Offline Offline
Full Member
***
Karma: 2
Posts: 131
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Alert: Make sure you download the latest one, the earlier version had a tricky bug that I have eliminated
DebugManchester.ino at
https://github.com/robwlakes/ArduinoWeatherOS

Another thing that has come to my attention is that the cheaper 433MHz Rx's have slow AGC and you may find that a stronger TX closer to the RX will cause weaker signals from sensors to be lost as the AGC does not recover in time to give them a fair go (so to speak in ultra scientific lingo!)  Try the Dorji line: RF-DRA886RX-S Dorji 443MHZ 107 dBm ASK Receiver, SMD Package

Cheers, Rob
« Last Edit: July 28, 2014, 08:09:37 am by robwlakes » Logged

Learning Flute and C++, heading for a meltdown.

Offline Offline
Newbie
*
Karma: 0
Posts: 6
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello Rob,

Thank you for your invaluable help. Below is my working sketch with commentary in the header. I have had problems with range using my RX so will look into the one you have recommended.

https://github.com/AMcAnerney/Arduino-F007th-Sketches

So the next goals are:
Incorporate error checking to remove the 3% of bad readings
Incorporate a warning if a sensor does not communicate for an extended period of time
Stop printing the data twice (is there something really basic I have overlooked?)
Work on negative temperatures
Devise a way to store values and send them only once a minute on the serial (as a precursor to sending updates to a server later on)
Improve the reception range (probably a hardware issue)

Please let me know if you have any comments or suggestions

Andrew
« Last Edit: July 29, 2014, 11:42:06 am by Tops » Logged

Lake Tyers Beach
Offline Offline
Full Member
***
Karma: 2
Posts: 131
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Andrew,

What excellent progress  smiley  very good work.  Thank you for your comments.

I have a few suggestions to make and maybe answers for some of your questions.

I will email your code back to you.  I am just a beginner on GIT-Hub and not confident about the push/pull process.  However I won't attach it here and indirectly publish it myself, and you will obviously need to consider what you keep and what you alter.

There was a late edition to the Debug program (won't be the last I am sure!) that concerned checking the first zero and also making sure the header hits had been equaled or exceeded.  I have added that in.
Code:
if ((!firstZero)&&(headerHits>=headerBits))
It basically forces the header to be valid before the first zero is accepted.

Secondly I have also simplified the idea of how to discard the leading zero and one in a more general way (Thanks to your request  smiley ) track the variable "discards" and you will spot what I have done. However it means the first byte in the manchester[bank][0] array is where to begin for the data ie index=0.  You will need to alter your calculations for temp and hum as they will be one byte out now.

I  have also added a 1 second interrupt that calls a small routine that can trigger off a 1 minute output of your data.  Not original but you will find the acknowledgement in the code.

If you increase the number of bytes required until you stop getting data then you know will have to reduce the maxBytes by one and then you should be looking at all the stable data.  If you are getting two quick transmissions of the same packet then you will probably need to store each packet in a separate manchester[bank][byteNos] and then compare them.  I have inserted a bank check routine that will check all four banks so feel free to modify that as you need.  It may send the packets twice as some form of redundancy and still have a check sum as well.  Watch the byte you suspect is the checksum and see what its behaviour is with changing values in the rest of the bytes.  The Bios WS sent the packet 4 times as validation check, the Oregon Scientific had a simple arithmetic checksum based on nibbles. Ambient may just rely on a comparison of two packets.

If you intend have multiple sensors each with its own transmitter you will need some way of stopping the output until all of them have registered valid numbers, then after that, send output each minute.  Each Tx'er ORing a bit in a byte if successful will allow an easy check.  Now that you have a timer in there you can also set up a check for Tx'ers going "quiet" for too long as well.

You will probably need a more generalised way of calculating from the Manchester[bank] the values for the transmitter/sensors so that your output to the server will be more manageable?   I think you need to use an array of some sort.

These changes may get rid of the 3% bad readings, especially the validation of course.

The final caveat is all I have done is see if the program with my alterations compiles which is definitely no guarantee it is still working.  I am looking forward to see what you do now.  I am impressed and feel you are definitely on the homeward stretch to quite a useful package.

Cheers, Rob 

Logged

Learning Flute and C++, heading for a meltdown.

Pages: 1 [2]   Go Up
Jump to: