Arduino Forum

Topics => Device Hacking => Topic started by: pawpro on Jan 02, 2016, 09:48 pm

Title: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 02, 2016, 09:48 pm
Hi,
 
I got a couple of EW 99 from Oregon Scientific Elements and a receiver from the attachment. I also got RFM69hcw but more about this later.

I want to receive the data from the EW99's. When I can I can take it from there.

Sadly I suspect it is not transmitting on one of Oregon's protocols (V1, V2 or V3).
I'm attaching an SDR capture.
I've tried https://github.com/robwlakes/ArduinoWeatherOS which is by far the best work I've seen put together in a useful way. However I doesn't decode the signal nor do I suspect it can having had a look at the waveform i.e. there is no header like in the other protocols.
It also seems the sensors broadcast on 433.860 & 433.845 but I think the sensor receives it ok - I can seem to see the signal when dumped raw to serial (while clearly seeing the broadcast taking place using SDR).

Can anyone help in any way?

I wouldn't mind using the RFM69 but there seems to be more moving parts and less clean reusable code published (but I'm open to suggestions).

Many thanks!

Update:
No idea what I am doing (pattern matching and loosely referencing v1,v2,v2.1 & v3 protocol ref PDF).
Here I restarted one sensor after a reading while switching the channels.
Code: [Select]
Preamble: 000110000000 Sync: 1010 ID: 10001000 CH2: 1000 Roll: 00100000 Flag?: 0010 Data?: 100010001000000010000010101010101010000000100010001010001010100000100010001010100010100000001000100010101000
Preamble: 000110000000 Sync: 1010 ID: 10001000 CH1: 0010 Roll: 00101010 Flag?: 0010 Data?: 101010001000000010001000000010101010000000100010100010000000100000000010001010100010001010100010100000000000
Preamble: 000110000000 Sync: 1010 ID: 10001000 CH3: 1010 Roll: 00000010 Flag?: 1000 Data?: 000010001000000010001000001010101010000000100010000010101000001010100010001010100010001010001000000010000000


The problems of which I know I have are:
- Neither of the described Oregon Scientific protocols has preamble of 12 bits
- Flag is changing so it is not the flag?
- Sensor ID is wrong length but then the channel is quite probably correct. Also it is the same for 2 different sensors...

Basically I cannot seem to match any of the oregon scientific protocols to raw bits.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: robwlakes on Jan 04, 2016, 07:35 am
Hi PawPro,

First thanks for the compliment,  I agree with your first impressions, it looks like a different beast (maybe not designed in-house??) to V3, but the waveform evidence so far looks as though it will yield to further study.  The absence of a standard header of a series of '1's is not always necessary as any sequence in Manchester protocol should provide a steady signal bias around a middle level, as time spent at either On or Off will be equal.  RFM69 (http://jdesbonnet.blogspot.com.au/2014/12/experiments-with-rfm69-433mhz-digital.html) certainly looks interesting, though I have not any experience at all with them.

Sorry I can't be of any further help, but I will watch your progress with great interest, so I hope you have a break through soon.

Rob
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: Riva on Jan 04, 2016, 08:32 am
Looking at the audio trace and I don't think it is Manchester encoded but uses a variable carrier off time with a fixed width on time to denote 0/1 bits. The message start/sync seems to be a longer carrier on/off time.
Can you attach or link to the actual audio file you took? If we can get the on/off timings then knocking up a bit of code to extract the data stream should be easy. Then it's just a case of trying to decode the stream.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: robwlakes on Jan 04, 2016, 01:09 pm
Apologies if I am wrong here, but the waveform initially looked classic Manchester me.
Only two types of pulse lengths, one twice as long as the other and the twice as long pulses are present as both hi and lo levels. It looks promising.  However a quick analysis by hand would suggest strongly that is not Manchester.

If you look at my first graphic below of your first pulses in your Audacity screen grabs, and assuming positive polarity the downward heading lines indicate zeroes, but then a double length pulse occurs at a time that violates the Manchester waveform. The red line I have drawn is where it should have gone to swap to a One, and if it was supposed to be a Zero then it should have just repeated, the preceding two Zero patterns.

I also tried it with synching it with the rising levels and eventually it contradicted the waveform as well, indicated by the red lines with "?" beneath them.

 I am happy to stand corrected, but I think it is something else. I reckon Riva's theory is looking better.

Rob
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 04, 2016, 05:16 pm
I'll upload the waveforms but in the meantime here is the "raw" capture of timings.
I would discard first 5 and last 4 that brings them into consistently above 1800us to ~4030us.

Code: [Select]

704,52,676,80,520,1872,2036,3828,4000,1864,2048,1856,2060,1844,2056,1860,4004,1852,4020,1848,4008,1848,2056,1856,2060,1848,2056,1848,2064,1848,4012,1848,2068,1844,2060,1844,2060,1848,2064,1848,4012,1844,2068,1848,2056,1848,4016,1844,2060,1848,2084,1820,4032,1832,2068,1840,2068,1844,2060,1844,2072,1840,4016,1844,2072,1840,4012,1844,2068,1844,4024,1832,4024,1844,4020,1840,4024,1836,4016,1848,2076,1828,2076,1836,2064,1844,4024,1840,4024,1840,4020,1836,4020,1844,2068,1836,4024,1844,4020,1844,4020,1844,4008,1848,2072,1840,4032,1828,4024,1836,2072,1836,4020,1844,4028,1832,2068,1840,4020,1844,4024,1840,4024,1832,4020,1844,2072,1840,4012,1844,2076,1840,4024,1836,2064,1844,4024,1840,4016,1840,4020,1848,4020,1844,2064,1840,2068,1836,4020,1844,4020,1844,21824,92,12,588,

616,16,792,24,580,1876,2024,3840,4004,1852,2056,1860,2044,1860,2056,1852,4016,1848,4016,1844,4008,1856,2052,1852,2056,1852,2064,1848,2060,1844,4016,1852,2052,1848,2064,1844,2064,1852,2056,1848,4016,1848,2060,1844,2060,1844,4024,1844,2064,1840,2068,1844,4020,1840,2068,1844,2060,1844,2060,1848,2068,1844,4020,1836,2068,1844,4016,1840,2072,1844,4020,1844,4020,1836,4020,1844,4016,1848,4020,1836,2068,1848,2060,1844,2068,1836,4020,1848,4020,1840,4020,1840,4020,1844,2076,1828,4024,1840,4020,1844,4020,1844,4024,1836,2076,1836,4012,1844,4024,1840,2072,1832,4024,1844,4024,1840,2060,1844,4020,1844,4024,1840,4020,1836,4020,1848,2064,1836,4020,1848,2068,1836,4028,1836,2072,1840,4012,1844,4020,1844,4020,1848,4016,1836,2072,1844,2056,1848,4020,1844,4020,1844,22052,36,1316,12,

556,36,728,16,708,1864,2044,3820,4000,1856,2064,1852,2052,1852,2060,1844,4024,1840,4024,1844,4020,1832,2072,1844,4016,1840,2076,1836,4036,1828,2068,1836,2076,1832,2076,1832,4036,1828,2072,1836,2076,1828,2072,1840,4024,1840,4028,1832,4020,1840,2080,1832,4020,1840,2076,1828,2084,1824,2084,1828,4032,1832,2072,1832,2088,1824,2076,1828,2080,1828,4028,1832,4036,1828,4028,1836,4032,1832,4032,1828,2072,1836,2084,1828,2072,1832,4032,1832,2072,1832,4060,1800,2084,1832,4036,1816,4036,1828,4032,1832,2084,1832,4028,1824,4032,1832,4032,1832,2080,1832,2072,1832,2084,1828,4036,1820,2084,1828,4036,1828,4032,1828,4032,1828,2076,1832,4032,1836,4032,1828,4028,1828,4036,1828,2080,1832,2072,1832,4032,1832,2080,1832,4032,1824,4036,1824,2088,1824,2080,1824,4040,1824,2668,20,12,924,

140,1876,680,48,1312,1868,2032,3824,4000,1868,2044,1860,2056,1848,2072,1844,4008,1848,4016,1848,4024,1836,2064,1844,2080,1824,2076,1836,4024,1840,2068,1840,2064,1840,2080,1828,4028,1836,2072,1836,2068,1836,2080,1832,4032,1832,4028,1828,2080,1836,2076,1824,4040,1828,2076,1824,2084,1828,2076,1832,4032,1832,2088,1816,2088,1824,2072,1828,2088,1828,4028,1832,4036,1820,4044,1824,4036,1828,4028,1836,2072,1828,2092,1820,2084,1824,4036,1828,4028,1832,4032,1836,2076,1824,4036,1828,4032,1836,4036,1824,2080,1824,4036,1828,4032,1832,4028,1828,2096,1816,2076,1832,4032,1828,4044,1820,2084,1824,4032,1828,4036,1828,4052,1808,2092,1812,4032,1832,4048,1816,4032,1832,4052,1812,2076,1828,2080,1832,4036,1828,2076,1828,4036,1828,4040,1824,2076,1828,2088,1816,4036,1828,8,1412,16,16,

136,1872,544,32,1464,1868,2044,3824,3992,1864,2060,1852,2060,1844,2064,1844,4024,1840,4020,1844,4016,1840,2072,1840,2060,1848,2072,1828,4028,1840,2072,1836,2072,1836,2072,1832,4028,1836,2076,1836,2068,1836,2088,1816,4028,1836,4040,1824,2072,1836,2080,1832,4028,1832,2084,1820,2084,1820,2084,1832,4028,1836,2076,1828,2088,1824,2076,1832,2080,1820,4028,1836,4028,1836,4036,1828,4032,1832,4032,1824,2088,1824,2072,1832,2092,1820,4040,1824,4032,1824,4036,1828,2080,1832,4028,1828,4032,1832,4036,1824,2088,1824,4036,1824,4036,1828,4028,1836,2080,1824,2084,1828,4032,1828,4036,1828,2076,1828,4036,1828,4036,1828,4040,1824,2072,1832,4040,1824,4036,1828,4028,1828,4032,1832,2092,1820,2068,1836,4032,1832,2080,1820,4040,1824,4036,1832,2076,1832,2080,1828,4040,1824,-1664,12,16668,44,
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: Riva on Jan 04, 2016, 05:28 pm
Maybe something like attached though the 0/1 times may be inverted.
I would think section A is to allow the receiver AGC to settle before sending the sync (message start).
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 04, 2016, 06:16 pm
The waveforms attached. Riva how do you call this scheme - so that I can read up.
Many thanks!

Update:
I've extrapolated from your screen grab that the data can be represented just using the timings:
Two "pulses per bit".
1. Short pulse followed by a short pulse = 1,
2. Short pulse followed by a long pulse = 0.

Out comes (4 different captures):
Code: [Select]

1,1,1,0,0,0,1,1,1,1,0,1,1,1,1,0,1,1,0,1,1,0,1,1,1,1,0,1,0,1,0,0,0,0,0,1,1,1,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,1,0,1,0,1,0,0,0,0,1,1,0
1,1,1,0,0,0,1,1,1,1,0,1,1,1,1,0,1,1,0,1,1,0,1,1,1,1,0,1,0,1,0,0,0,0,0,1,1,1,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,1,0,1,0,1,0,0,0,0,1,1,0
1,1,1,0,0,0,1,1,1,0,1,1,1,0,1,1,1,0,0,1,1,0,1,1,1,0,1,1,1,1,0,0,0,0,0,1,1,1,0,0,0,1,0,0,0,1,0,0,0,1,1,0,0,1,0,0,0,1,0,0,0,0,1,1,0,1,0,0,1,1
1,1,1,0,0,0,1,1,1,0,1,1,1,0,1,1,1,0,0,1,1,0,1,1,1,0,1,1,1,1,0,0,0,0,0,1,1,1,0,0,0,1,0,0,0,1,0,0,0,1,1,0,0,1,0,0,0,1,0,0,0,0,1,1,0,1,0,0,1,1
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: Riva on Jan 05, 2016, 08:53 am
Attached is some untested code that may decode your signal.
I would like to test it with the samples you supplied before posting but time is against me and it may be the weekend before I could do that so maybe you can try it and see.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 05, 2016, 11:16 am
@Riva: quick feedback. That is more substantial than I would have expected! Thank you very much.

With line 104:
Code: [Select]
     
      }
      rise_Time = uTime;                                  // Store rise time
    }


changed such that:

Code: [Select]
     
      }
      rise_Time = uTime + rise_Time;                      // Store rise time
    }


It does indeed detect and confirms the previous analysis.
The change may not follow logically in what we are doing but it follows your use of uTime. After getting a bit further I will rewrite this bit. For now I'll let the data flow.
Code: [Select]
We have at least 2 consecutive matching reads
11100011110100101001101110011010000111000010110101100100011001001111011
We have at least 2 consecutive matching reads
11100011110100101001101110011010000111000010110101100100011001001111011
We have at least 2 consecutive matching reads
11100011110100101001101110011110000111000010110101100100011000001011011
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: robwlakes on Jan 05, 2016, 12:41 pm
Just for interest sake I have written (ahem, adapted an older program) Python program to draw the timing data in a previous post (04-01-2016, 16:16:12).  The high/Low representation is just a guess at the moment. But the overall patterns may be clearer with this than from the Audacity recording.

If you would like the Python program I will clean it up and post it here as well.

You appear to have nailed it already though I see :-)

Cheers, Rob
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: Riva on Jan 05, 2016, 12:54 pm
Thanks for pointing out my programming error pawpro, I have altered my local copy to match.
I'm amazed it (almost) worked out of the box. Maybe I should pick some lottery numbers today  :)
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 06, 2016, 04:33 pm
I will be back on this within 24h. But today I need to integrate Audeme's Movi board that just arrived.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 08, 2016, 12:55 am
I'm a bit lost. I don't know if I'm doing this right.
Assuming the data capturing and decoding is right (I'm starting to think it may not be).

I'm trying to convert the binary data to dec and spot the patterns. Sadly the sensor doesn't have a screen and I don't have the weather station. So I'm using other sensors to get the readouts.
This is what I have. I am breaking the stream in to 4 bit chunks (4, 8 and 12) put together and converting it to decimal looking for my temp and humidity:
Code: [Select]
21.2C, 80%
11100010101110101111101111011000000111010100010100000100001001110000110:
 0. 1110   14;
 1. 0010    2; 11100010  226;
 2. 1011   11; 00101011   43; 111000101011 3627;
 3. 1010   10; 10111010  186; 001010111010  698;
 4. 1111   15; 10101111  175; 101110101111 2991;
 5. 1011   11; 11111011  251; 101011111011 2811;
 6. 1101   13; 10111101  189; 111110111101 4029;
 7. 1000    8; 11011000  216; 101111011000 3032;
 8. 0001    1; 10000001  129; 110110000001 3457;
 9. 1101   13; 00011101   29; 100000011101 2077;
10. 0100    4; 11010100  212; 000111010100  468;
11. 0101    5; 01000101   69; 110101000101 3397;
12. 0000    0; 01010000   80; 010001010000 1104;
13. 0100    4; 00000100    4; 010100000100 1284;
14. 0010    2; 01000010   66; 000001000010   66;
15. 0111    7; 00100111   39; 010000100111 1063;
16. 0000    0; 01110000  112; 001001110000  624;

11100010101110101111101111010100000111010100010100000100001010110101010:
 0. 1110   14;
 1. 0010    2; 11100010  226;
 2. 1011   11; 00101011   43; 111000101011 3627;
 3. 1010   10; 10111010  186; 001010111010  698;
 4. 1111   15; 10101111  175; 101110101111 2991;
 5. 1011   11; 11111011  251; 101011111011 2811;
 6. 1101   13; 10111101  189; 111110111101 4029;
 7. 0100    4; 11010100  212; 101111010100 3028;
 8. 0001    1; 01000001   65; 110101000001 3393;
 9. 1101   13; 00011101   29; 010000011101 1053;
10. 0100    4; 11010100  212; 000111010100  468;
11. 0101    5; 01000101   69; 110101000101 3397;
12. 0000    0; 01010000   80; 010001010000 1104;
13. 0100    4; 00000100    4; 010100000100 1284;
14. 0010    2; 01000010   66; 000001000010   66;
15. 1011   11; 00101011   43; 010000101011 1067;
16. 0101    5; 10110101  181; 001010110101  693;

38.5C, 20%
11100011101100010011100101011110000111000100111011000110101000001010011:
 0. 1110   14;
 1. 0011    3; 11100011  227;
 2. 1011   11; 00111011   59; 111000111011 3643;
 3. 0001    1; 10110001  177; 001110110001  945;
 4. 0011    3; 00010011   19; 101100010011 2835;
 5. 1001    9; 00111001   57; 000100111001  313;
 6. 0101    5; 10010101  149; 001110010101  917;
 7. 1110   14; 01011110   94; 100101011110 2398;
 8. 0001    1; 11100001  225; 010111100001 1505;
 9. 1100   12; 00011100   28; 111000011100 3612;
10. 0100    4; 11000100  196; 000111000100  452;
11. 1110   14; 01001110   78; 110001001110 3150;
12. 1100   12; 11101100  236; 010011101100 1260;
13. 0110    6; 11000110  198; 111011000110 3782;
14. 1010   10; 01101010  106; 110001101010 3178;
15. 0000    0; 10100000  160; 011010100000 1696;
16. 1010   10; 00001010   10; 101000001010 2570;

11100010101100010011100011111000000111010100111011000111000001100111111:
 0. 1110   14;
 1. 0010    2; 11100010  226;
 2. 1011   11; 00101011   43; 111000101011 3627;
 3. 0001    1; 10110001  177; 001010110001  689;
 4. 0011    3; 00010011   19; 101100010011 2835;
 5. 1000    8; 00111000   56; 000100111000  312;
 6. 1111   15; 10001111  143; 001110001111  911;
 7. 1000    8; 11111000  248; 100011111000 2296;
 8. 0001    1; 10000001  129; 111110000001 3969;
 9. 1101   13; 00011101   29; 100000011101 2077;
10. 0100    4; 11010100  212; 000111010100  468;
11. 1110   14; 01001110   78; 110101001110 3406;
12. 1100   12; 11101100  236; 010011101100 1260;
13. 0111    7; 11000111  199; 111011000111 3783;
14. 0000    0; 01110000  112; 110001110000 3184;
15. 0110    6; 00000110    6; 011100000110 1798;
16. 0111    7; 01100111  103; 000001100111  103;

-6.4C, 80%
11100010101010111111111101001100000111010101010000000000101100100001010:
 0. 1110   14;
 1. 0010    2; 11100010  226;
 2. 1010   10; 00101010   42; 111000101010 3626;
 3. 1011   11; 10101011  171; 001010101011  683;
 4. 1111   15; 10111111  191; 101010111111 2751;
 5. 1111   15; 11111111  255; 101111111111 3071;
 6. 0100    4; 11110100  244; 111111110100 4084;
 7. 1100   12; 01001100   76; 111101001100 3916;
 8. 0001    1; 11000001  193; 010011000001 1217;
 9. 1101   13; 00011101   29; 110000011101 3101;
10. 0101    5; 11010101  213; 000111010101  469;
11. 0100    4; 01010100   84; 110101010100 3412;
12. 0000    0; 01000000   64; 010101000000 1344;
13. 0000    0; 00000000    0; 010000000000 1024;
14. 1011   11; 00001011   11; 000000001011   11;
15. 0010    2; 10110010  178; 000010110010  178;
16. 0001    1; 00100001   33; 101100100001 2849;


Any ideas?

@Riva: I'm attaching further "improved" version of your sketch. If you can look pass the styling changes I "needed" there were couple of "serious" issues one with delay(100) being executed every loop() invocation, sync pulse detection was triggered when the pulse was longer than but not "smaller than" so was triggered very often (clearing the previous data - also changed). I've broken the debug LED a bit. 
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Jan 08, 2016, 03:51 am
I suggest to solve the temperature encoding first. Collect data at a bunch of different temperatures, trying to measure those as accurately as possible, and look for a pattern. It is very helpful to have data for temperatures above and below freezing, and to have temperatures separated by one or more single digit changes, like 20.0, 20.1 and 20.2 C.

I've decoded a number of sensors and there are two common approaches for storing temperature: always Celcius, often in tenths of a degree C, and represented by either BCD digits or binary (10-12 bits). Sometimes there is an offset so that the number is positive (e.g. 50, or 500).

So, I would look for a bit field that shows changes consistent with increasing temperature, with one of those two possibilities in mind.

This encoding is a real departure from Oregon Scientific's previous data encoding schemes (which used BCD digits), so it may or may not be very helpful to consider them.

Also, keep in mind that most of these sensors are accurate to around +/- 1 degree C, so give some leeway when comparing your own measurements.

If you like, post a table of complete messages, along with the estimated temperature, and others will be happy to help solve it. What you posted above, with the guessed breakdowns, is confusing and has too few data points for me to make sense of.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 08, 2016, 03:56 am
@jremington thanks for your comments! I will have to set up a more controlled test. I thought they were a fairly recent model but it seems they not. So perhaps this is the old way for OS or maybe as someone suggested it's been done by another manufacturer.

Do you think the binary data we capture is correct?
Does anyone have any doubts?

Here is what you asked for. I will try to gather better data soon.
Only unique entries for past couple of hours. On a heater next to another sensor which reports 39.9C/20%

Code: [Select]

11100011101100010011100101110000000111000100111011000110100011110110111
11100011101100010011100101011110000111000100111011000110101000001010011
11100010101100010011100011111000000111010100111011000111000001100111111
11100010101100010011100011110110000111010100111011000111000010001000010
11100010101100010011100011110100000111010100111011000111000010101010011
11100010101100010011100011110010000111010100111011000111000011001100010
11100010101100010011100011110000000111010100111011000111000011101110011
11100010101100010011100011101110000111010100111011000111000100010001110
11100010101100010011100011101100000111010100111011000111000100110011111
11100010101100010011100011011110000111010100111011000111001000000011111
11100010101100010011100011011100000111010100111011000111001000100100000
11100010101100010011100011011000000111010100111011000111001001101000000
11100010101100010011100011010100000111010100111011000111001010101100100
11100010101100010011100011010010000111010100111011000111001011001110011
11100010101100010011100011010000000111010100111011000111001011110000100
11100010101100010011100011001110000111010100111011000111001100010011111
11100010101100010011100011010110000111010100111011000111001010001010011
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: mauried on Jan 08, 2016, 05:02 am
Sometimes it helps with reverse decoding of unknown sensors, to find out all the types of sensors that the receiving console supports, including ones that you dont have, as the data fields being transmitted will be common to all the sensors.
You can usually look up the accuracy specs for the temps and the RH and find out if its 1 Deg C or 0.1 D C
and similar with the RH.
Are these sensors actually made by OS or are they 3rd party sensors which have been re badged as OS ones?
The latter would explain the non standard protocol.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Jan 08, 2016, 07:09 am
I'll bet you are right that the EW99 is a 3rd party sensor.
It doesn't seem to be available in the U.S., which is rather strange for a U.S. company!
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 08, 2016, 07:15 am
well http://stackoverflow.com/questions/12015112/checksum-crc-calculation
This guy shared his findings (while looking for CRC calc help) and...
It seems I am one bit off. When I skip one bit things look a bit more sensible. Still confirming.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: Riva on Jan 08, 2016, 08:48 am
To determine the most likely range of bits used for temperature & humidity place the sensor in the fridge/freezer (depending on it's specs) for about 30 minutes then bring it out and start logging the data. The temperature, humidity & checksum bits should be changing quite rapidly while fixed values like sensor ID, channel number will remain constant.
Once you have an idea what bits to look at you might have a better chance of decoding them.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Jan 08, 2016, 06:14 pm
I don't think that the sensor described in the stackoverflow post is of the same type.

I've rearranged the data table from post#14 into 8-bit columns as follows and see that primarily, fields  D, H and I change. B and F take on just two values and the same row in each changes. Presumably column I (or perhaps H and I) is a checksum.

If D and H are temperature and humidity, they change a lot. This is inconsistent with your claim that the temperature and humidity are relatively constant.

I second Riva's suggestion of the freezer.

Code: [Select]
A       B        C        D        E        F        G        H        I
1110001 11011000 10011100 10111000 00001110 00100111 01100011 01000111 10110111
1110001 11011000 10011100 10101111 00001110 00100111 01100011 01010000 01010011
1110001 01011000 10011100 01111100 00001110 10100111 01100011 10000011 00111111
1110001 01011000 10011100 01111011 00001110 10100111 01100011 10000100 01000010
1110001 01011000 10011100 01111010 00001110 10100111 01100011 10000101 01010011
1110001 01011000 10011100 01111001 00001110 10100111 01100011 10000110 01100010
1110001 01011000 10011100 01111000 00001110 10100111 01100011 10000111 01110011
1110001 01011000 10011100 01110111 00001110 10100111 01100011 10001000 10001110
1110001 01011000 10011100 01110110 00001110 10100111 01100011 10001001 10011111
1110001 01011000 10011100 01101111 00001110 10100111 01100011 10010000 00011111
1110001 01011000 10011100 01101110 00001110 10100111 01100011 10010001 00100000
1110001 01011000 10011100 01101100 00001110 10100111 01100011 10010011 01000000
1110001 01011000 10011100 01101010 00001110 10100111 01100011 10010101 01100100
1110001 01011000 10011100 01101001 00001110 10100111 01100011 10010110 01110011
1110001 01011000 10011100 01101000 00001110 10100111 01100011 10010111 10000100
1110001 01011000 10011100 01100111 00001110 10100111 01100011 10011000 10011111
1110001 01011000 10011100 01101011 00001110 10100111 01100011 10010100 01010011
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 08, 2016, 06:30 pm
It's been outside for past 8 hours. Temperature reported by another sensor is 1.2C.
Readings for the past 5 min.

Code: [Select]
11100010101111100111111111111100000111010100000110000000000000101011001
11100010101111100111111111111100000111010100000110000000000000101011001
11100010101111100111111111111100000111010100000110000000000000101011001
11100010101111100111111111111100000111010100000110000000000000101011001
11100010101111100111111111111100000111010100000110000000000000101011001


I've opened one of these out. Photo attached in case you recognise something (not much to see).


UPDATE:
Well the temperature just rose by about 0.1 degree outside and something interesting (perhaps happened).

This is the transition afait
Code: [Select]
11100010101111100111111111111100000111010100000110000000000000101011001
11100010101111100111111111111000000111010100000110000000000001101111001


What are they doing?

By just adding some spaces I get the following:
Code: [Select]

11   1000 1010 1111 1001 1111 1111 1111   0000
     0111 0101 0000 0110 0000 0000 0000   101011001

11   1000 1010 1111 1001 1111 1111 1110   0000
     0111 0101 0000 0110 0000 0000 0001   101111001

Looks like somewhere mid way they reverse the bits and transmit again?

UPDATE 2:
Some old captures broken down in exactly the same way
Code: [Select]

11 1000111101001010011011100110 1000
   0111000010110101100100011001 001111011

11 1000111101001010011011100110 1000
   0111000010110101100100011001 001111011

11 1000111101001010011011100110 1000
   0111000010110101100100011001 001111011

11 1000111101001010011011100111 1000
   0111000010110101100100011000 001011011
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Jan 08, 2016, 09:00 pm
Your results suggest that the following 12 bits are the temperature, a signed 12 bit integer in tenths of a degree C. The second line is the complement of the data (a common error-checking approach).

In this interpretation, the top temperature is negative (-1 or -0.1 degree) and then changes to -0.2. Or, the bottom code is the temperature 0.0, which then changes to 0.1.
Code: [Select]

1111 1111 1111
0000 0000 0000

1111 1111 1110
0000 0000 0001


If this is correct, the 16 bits preceding the temperature could be the humidity, and perhaps some other measure.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Jan 09, 2016, 06:47 am
I retract post #21 (although the observation is valid), because I think I found the temperature, encoded as BCD digits.

I converted the last 6 columns of the data in post #14 and earlier ones (with data for 21.2, around 0.0 and -6.5) to hex.  You mentioned in post #14 that the temperature was near 39 C. The last 3 hex digits before the presumed 8-bit checksum appear to be BCD digits for temperatures in the range of 38.3 and 39.8

If you add in the data for 21.2, about 0.0 and -6.5 degrees, it becomes rather convincing, but considering that -6.5 is represented as 059, there must be a negative sign somewhere else.

The complement of the data comes earlier in the message. I don't know what that means.

Below, the presumed temperature digits are separated by spaces for clarity.
Code: [Select]

about 38 C
1110001 11011000 10011100 10111000 00001110 00100111 01100011 01000111 10110111 B80E276 347 B7
1110001 11011000 10011100 10101111 00001110 00100111 01100011 01010000 01010011 AF0E276 350 53
1110001 01011000 10011100 01111100 00001110 10100111 01100011 10000011 00111111 7C0EA76 383 3F
1110001 01011000 10011100 01111011 00001110 10100111 01100011 10000100 01000010 7B0EA76 384 42
1110001 01011000 10011100 01111010 00001110 10100111 01100011 10000101 01010011 7A0EA76 385 53
1110001 01011000 10011100 01111001 00001110 10100111 01100011 10000110 01100010 790EA76 386 62
1110001 01011000 10011100 01111000 00001110 10100111 01100011 10000111 01110011 780EA76 387 73
1110001 01011000 10011100 01110111 00001110 10100111 01100011 10001000 10001110 770EA76 388 8E
1110001 01011000 10011100 01110110 00001110 10100111 01100011 10001001 10011111 760EA76 389 9F
1110001 01011000 10011100 01101111 00001110 10100111 01100011 10010000 00011111 6F0EA76 390 1F
1110001 01011000 10011100 01101110 00001110 10100111 01100011 10010001 00100000 6E0EA76 391 20
1110001 01011000 10011100 01101100 00001110 10100111 01100011 10010011 01000000 6C0EA76 393 40
1110001 01011000 10011100 01101010 00001110 10100111 01100011 10010101 01100100 6A0EA76 395 64
1110001 01011000 10011100 01101001 00001110 10100111 01100011 10010110 01110011 690EA76 396 73
1110001 01011000 10011100 01101000 00001110 10100111 01100011 10010111 10000100 680EA76 397 84
1110001 01011000 10011100 01100111 00001110 10100111 01100011 10011000 10011111 670EA76 398 9F
1110001 01011000 10011100 01101011 00001110 10100111 01100011 10010100 01010011 6B0EA76 394 53

about 21.2 C
1110001 01011101 01111101 11101100 00001110 10100010 10000010 00010011 10000110 EC0EA28 213 86
1110001 01011101 01111101 11101010 00001110 10100010 10000010 00010101 10101010 EA0EA28 215 AA

about -6.5 C
1110001 01010101 11111111 10100110 00001110 10101010 00000000 01011001 00001010 A60EAA0 059 0A

about 1.2
1110001 01011111 00111111 11111110 00001110 10100000 11000000 00000001 01011001 7E0EA0C 001 59

about 0.0
1110001 01011111 00111111 11111110 00001110 10100000 11000000 00000001 01011001 FE0EA0C 001 59
1110001 01011111 00111111 11111100 00001110 10100000 11000000 00000011 01111001 FC0EA0C 003 79
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: Riva on Jan 09, 2016, 08:32 am
There is always the possibility the reading program I knocked up is interpreting the bits the wrong way round (I was guessing) so 6000us is a zero and 4000us  is a 1

Code: [Select]
//    ______        ______     ___     ___
//   |      |      |      |   |   |   |
//___|      |______|      |___|   |___|
//
//   |    Sync     |     1    |   0   |
//   |    8000us   |  6000us  | 4000us
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Jan 09, 2016, 05:16 pm
If you invert all the bits in the message, the presumed temperature field just comes earlier in the message. Converting the central 48 bits (which appear to carry the data and the data complement) to hex, you get

10011100 01101011 00001110 10100111 01100011 10010100 =
9C6B0EA76394, which complemented, is
6394F1589C6B

Either way, the proposed temperature digits would be 394.

Assuming 1=4000 us and 0 = 6000 us is more likely to be correct, as people often send the data and then the data complement as a check.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: robwlakes on Jan 10, 2016, 07:47 am
Hi folks,

You may like to process the packet trough the following routine as it breaks the packet up into binary and hexadecimal.

//Useful to invoke to debug the byte Array
void hexBinDump() {
  //Serial.println("T A3 10100011 07 00000111 02 00000010 AA 10101010 F0 11110000 06 00000110 FF 11111111 07 00000111 33 00110011 60 01100000");
  Serial.print("D ");
  for ( int i = 0; i < maxBytes; i++) {
    byte mask = B10000000;
    if (manchester < 16) {
      Serial.print("0");
    }
    Serial.print(manchester, HEX);
    Serial.print(" ");
    for (int k = 0; k < 8; k++) {
      if (manchester & mask) {
        Serial.print("1");
      }
      else {
        Serial.print("0");
      }
      mask = mask >> 1;
    }
    Serial.print(" ");
  }
  Serial.println();
}


This may help you to spot either binary patterns or BCD patterns in your data.  How relevant this is will depend on how the byte patterns line up, so you may have to shift your primary data capturing process along or back a few bits to get anything meaningful from it.  However putting a standard alcohol analogue themometer beside the device and getting a ball park figure to look for may well help.
Also don't be too surprised if for one factor eg Temperature they use binary representation of temperature byte1*10+byte2=temp/10 and for Humidity the use BCD B00110111 ie 37%.  Don't assume too much and keep an open mind.

Cheers, R
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: pawpro on Jan 13, 2016, 07:42 am
BCD as in each i.e. 4 bits represent a single digit left to right rather than larger i.e. 12 bits being just base converted to decimal... right? In that case <insert facepalm>. I wrote a "walking" decoder over a CSV of bit dumps but have not found a reliable representation probably because BCD thing... ehh. Will have a look now.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: robwlakes on Jan 13, 2016, 11:53 am
Hi PawPro,

BCD are 4 bit numbers, so 2 independent BCD's can be in one 8 bit number:

eg 8bits -> 10010011, is 93 in  BCD, ie 8+0+0+1, 0+0+2+1

but as pure binary 128+0+0+16+0+0+2+1= 147 as in common base 10.

As hexadecimal decodes BCD if it exists, just doing Hexadecimal is also handy, as any illegal BCD such as 1100, ie  C in the data stream will be obvious too.

Rob
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Jan 13, 2016, 06:25 pm
I think that all Oregon Scientific protocols use BCD data encoding, but this particular sensor may be from a third party. Nevertheless, BCD is very commonly used.
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: robwlakes on Jan 14, 2016, 04:29 am
Yes I agree, all the OS V3.0 is BCD.  However a couple of little tricks to the data packet that made it a bit more difficult to decode was the "synch" bit was inside the first byte (and nybble) boundaries, but less obvious was the nybbles had the order of bits reversed eg

one nybble -
MSB            LSB
  A    B   C   D

was transmitted to the receiver as DCBA.  So  the receiving software had to reverse the bits back to ABCD
(where A=8, B=4, C=2 and D=1) then start processing to find the value. eg

  temperature = (double)((nyb(11) * 100) + (nyb(10) * 10) + nyb(9)) / 10;

However the sequence of nibbles of BCD were in ascending order after that and easy to work with.

A quick hand check of where the byte boundaries are, and the possibility of nybble reversal may be worth while.

Cheers, Rob
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: AlexeyRU on Mar 04, 2016, 01:51 am
Hi PawPro,

If this subject is still interesting to you.

I have an ew99 sensor and a base station of ew91 which shows temperature. I obtain data from the sensor on the SDR receiver and I use rtl_433 for decoding of a signal (https://github.com/merbanan/rtl_433 (https://github.com/merbanan/rtl_433)).

Analyzing pulses...
Total count:   75,  width: 91787                (367.1 ms)
Pulse width distribution:
 [ 0] count:   74,  width:   474 [466;482]      (1896 us)
 [ 1] count:    1,  width:   968 [968;968]      (3872 us)
Gap width distribution:
 [ 0] count:   36,  width:   501 [497;508]      (2004 us)
 [ 1] count:   38,  width:   990 [984;998]      (3960 us)

Sensors have the similar protocol
In nexus.c
 * the packets are ppm modulated (distance coding) with a pulse of ~500 us
 * followed by a short gap of ~1000 us for a 0 bit or a long ~2000 us gap for a
 * 1 bit, the sync gap is ~4000 us.
In prologue.c
 * the packets are ppm modulated (distance coding) with a pulse of ~500 us
 * followed by a short gap of ~2000 us for a 0 bit or a long ~4000 us gap for a
 * 1 bit, the sync gap is ~9000 us.
Probably?

short gap for a 0 bit
long  gap for a 1 bit

You decoded a signal on the contrary - a short gap 1, a long gap - 0.

If to make on the contrary (see attach file), then we will receive:
Code: [Select]

10001110_10010100_11001010_00101000_1111000.....
8___e____9___4____c___a____2___8____f

10001110_00010100_11000010_00100111_1111000.....
8___e____1___4____c___2____2___7____f

10001110_00010100_11000010_00100111_1111000.....
8___e____1___4____c___2____2___7____f

10001110_10010011_01111010_00100100_1111000.....
8___e____9___3____7___a____2___4____f


For my device I have:
Code: [Select]

EW99 bitbuffer event 2016-02-29 23:19:51 - 23,8 C
10001110 10010000 11000010 00111000 11110001 01101111 00111101 11000111 11110101
8___e____9___0____c___2____3___8
EW99 bitbuffer event 2016-03-01 02:11:49 - 0,0 C
10001110 10010000 11000000 00000000 11110001 01101111 00111111 11111111 11000110
8___e____9___0____c___0____0___0
EW99 bitbuffer event 2016-03-01 02:52:49 - -0,6 C
10001110 10011000 11000000 00000110 11110001 01100111 00111111 11111001 11100010
8___e____9___8____c___0____0___6


EW99 bitbuffer event 2016-02-29 16:46:03 - 19,4 C canal 1
10001110 00010011 10010001 10010100 11110001 11101100 01101110 01101011 01100010
8___e____1___3____9___1____9___4
EW99 bitbuffer event 2016-02-29 16:44:22 - 19,3 C canal 2
10001110 00100010 10010001 10010011 11110001 11011101 01101110 01101100 01110111
8___e____2___2____9___1____9___3
EW99 bitbuffer event 2016-02-29 16:38:48 - 19,4 C canal 3
10001110 00110110 01110001 10010100 11110001 11001001 10001110 01101011 00111011
8___e____3___6____7___1____9___4

EW99 bitbuffer event 2016-02-29 15:45:01 - reverse the bits
10001110 10010000 00000001 10010011 11110001 01101111 11111110 01101100 10110111
_1110001 01101111 11111110 01101100__==================================


So, that as a result:
Code: [Select]

8___e____3___6____7___1____9___4
10001110 00110110 01110001 10010100 11110001 11001001 10001110 01101011 00111011
___________^^   canal___^^_^^^^^^^^ temp                                           CRS?
_____________^  sign


It is all so far.

Cheers, Alex
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Mar 04, 2016, 04:56 am
That appears to agree with my previous analysis, with the bits inverted. The temp is BCD encoded, and sent along with its complement later in the message.

Quote
If you invert all the bits in the message, the presumed temperature field just comes earlier in the message. Converting the central 48 bits (which appear to carry the data and the data complement) to hex, you get

10011100 01101011 00001110 10100111 01100011 10010100 =
9C6B0EA76394, which complemented, is
6394F1589C6B  (temp = 39.4 C)
Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: AlexeyRU on Mar 05, 2016, 02:06 am
Hi, jremington!

I agree with "bits inverted" excepting the first bit first four bytes.
Code: [Select]

EW99 bitbuffer event 2016-02-29 15:45:01 - reverse the bits (excepting the first bit of first byte)
10001110 10010000 00000001 10010011 11110001 01101111 11111110 01101100 10110111
_1110001 01101111 11111110 01101100__==================================


I agree with "the temp is BCD encoded" - but "packed BCD" (https://en.wikipedia.org/wiki/Binary-coded_decimal (https://en.wikipedia.org/wiki/Binary-coded_decimal).

I have been surprised - "(temp = 39.4 C)" in your message. It in Africa :) ? Or in the room :o?

Ok.

The sensor transfers temperature from-40 to +60 (according to the description).
I have put the sensor on the battery and have heated to 45,5 C
Code: [Select]
EW99 bitbuffer event 2016-03-05 03:27:52 - 45,5 C
10001110 10010000 11000100 01010101 11110001 01101111 00111011 10101010 11101000
8___e___ 9___0___ c___4___ 5___5___ f1 6f 3b aa e8


So, three bits on tens of temperature at the end of the third byte, the fourth byte of unit + 1/10.

As a result what we have:
Code: [Select]
EW99 bitbuffer event 2016-03-05 03:27:52 - 45,5 C
10001110 10010000 11000100 01010101 11110001 01101111 00111011 10101010 11101000
kkkkkkkk ??ccs??? ?????ttt tttttttt _rrrrrrr rrrrrrrr rrrrrrrr rrrrrrrr CRS__CRS

k - device code
c - canal
s - sign
t - temperature
r - reverse (excepting the first bit)
? - ???????


How to define what is meant by the remained unknown bits? There are ideas?

Yours faithfully, Alex




Title: Re: 433 Weather sensor from Oregon Scientific problems decoding
Post by: jremington on Mar 05, 2016, 05:54 am
Quote
I have been surprised - "(temp = 39.4 C)" in your message. It in Africa :) ? Or in the room :o?
Read post #14.

Quote
How to define what is meant by the remained unknown bits?
"CRS" could be a CRC checksum, but there are other possibilities, like a simple sum. Other bits could be battery status, temperature increasing or decreasing, etc.

You have to look at a lot of entries to make sure, and if "CRS" is a CRC checksum, many bits will change, even if only one bit in the message changes.

If it is a CRC you need to determine the polynomial and input/output constants. I have used the program reveng to reverse engineer CRC calculations. http://reveng.sourceforge.net/