433 Weather sensor from Oregon Scientific problems decoding

[pawpro]

Hi,
 
I got a couple of EW 99 from Oregon Scientific Elements and a receiver from the attachment. I also got RFM69hcw but more about this later.

I want to receive the data from the EW99's. When I can I can take it from there.

Sadly I suspect it is not transmitting on one of Oregon's protocols (V1, V2 or V3).
I'm attaching an SDR capture.
I've tried https://github.com/robwlakes/ArduinoWeatherOS which is by far the best work I've seen put together in a useful way. However I doesn't decode the signal nor do I suspect it can having had a look at the waveform i.e. there is no header like in the other protocols.
It also seems the sensors broadcast on 433.860 & 433.845 but I think the sensor receives it ok - I can seem to see the signal when dumped raw to serial (while clearly seeing the broadcast taking place using SDR).

Can anyone help in any way?

I wouldn't mind using the RFM69 but there seems to be more moving parts and less clean reusable code published (but I'm open to suggestions).

Many thanks!

Update:
No idea what I am doing (pattern matching and loosely referencing v1,v2,v2.1 & v3 protocol ref PDF).
Here I restarted one sensor after a reading while switching the channels.

Code: [Select]

Preamble: 000110000000 Sync: 1010 ID: 10001000 CH2: 1000 Roll: 00100000 Flag?: 0010 Data?: 100010001000000010000010101010101010000000100010001010001010100000100010001010100010100000001000100010101000
Preamble: 000110000000 Sync: 1010 ID: 10001000 CH1: 0010 Roll: 00101010 Flag?: 0010 Data?: 101010001000000010001000000010101010000000100010100010000000100000000010001010100010001010100010100000000000
Preamble: 000110000000 Sync: 1010 ID: 10001000 CH3: 1010 Roll: 00000010 Flag?: 1000 Data?: 000010001000000010001000001010101010000000100010000010101000001010100010001010100010001010001000000010000000


The problems of which I know I have are:
- Neither of the described Oregon Scientific protocols has preamble of 12 bits
- Flag is changing so it is not the flag?
- Sensor ID is wrong length but then the channel is quite probably correct. Also it is the same for 2 different sensors...

Basically I cannot seem to match any of the oregon scientific protocols to raw bits.

[robwlakes]

Hi PawPro,

First thanks for the compliment,  I agree with your first impressions, it looks like a different beast (maybe not designed in-house??) to V3, but the waveform evidence so far looks as though it will yield to further study.  The absence of a standard header of a series of '1's is not always necessary as any sequence in Manchester protocol should provide a steady signal bias around a middle level, as time spent at either On or Off will be equal.  RFM69 certainly looks interesting, though I have not any experience at all with them.

Sorry I can't be of any further help, but I will watch your progress with great interest, so I hope you have a break through soon.

Rob

[Riva]

Looking at the audio trace and I don't think it is Manchester encoded but uses a variable carrier off time with a fixed width on time to denote 0/1 bits. The message start/sync seems to be a longer carrier on/off time.
Can you attach or link to the actual audio file you took? If we can get the on/off timings then knocking up a bit of code to extract the data stream should be easy. Then it's just a case of trying to decode the stream.

[robwlakes]

Apologies if I am wrong here, but the waveform initially looked classic Manchester me.
Only two types of pulse lengths, one twice as long as the other and the twice as long pulses are present as both hi and lo levels. It looks promising.  However a quick analysis by hand would suggest strongly that is not Manchester.

If you look at my first graphic below of your first pulses in your Audacity screen grabs, and assuming positive polarity the downward heading lines indicate zeroes, but then a double length pulse occurs at a time that violates the Manchester waveform. The red line I have drawn is where it should have gone to swap to a One, and if it was supposed to be a Zero then it should have just repeated, the preceding two Zero patterns.

I also tried it with synching it with the rising levels and eventually it contradicted the waveform as well, indicated by the red lines with "?" beneath them.

 I am happy to stand corrected, but I think it is something else. I reckon Riva's theory is looking better.

Rob

[pawpro]

I'll upload the waveforms but in the meantime here is the "raw" capture of timings.
I would discard first 5 and last 4 that brings them into consistently above 1800us to ~4030us.

Code: [Select]


704,52,676,80,520,1872,2036,3828,4000,1864,2048,1856,2060,1844,2056,1860,4004,1852,4020,1848,4008,1848,2056,1856,2060,1848,2056,1848,2064,1848,4012,1848,2068,1844,2060,1844,2060,1848,2064,1848,4012,1844,2068,1848,2056,1848,4016,1844,2060,1848,2084,1820,4032,1832,2068,1840,2068,1844,2060,1844,2072,1840,4016,1844,2072,1840,4012,1844,2068,1844,4024,1832,4024,1844,4020,1840,4024,1836,4016,1848,2076,1828,2076,1836,2064,1844,4024,1840,4024,1840,4020,1836,4020,1844,2068,1836,4024,1844,4020,1844,4020,1844,4008,1848,2072,1840,4032,1828,4024,1836,2072,1836,4020,1844,4028,1832,2068,1840,4020,1844,4024,1840,4024,1832,4020,1844,2072,1840,4012,1844,2076,1840,4024,1836,2064,1844,4024,1840,4016,1840,4020,1848,4020,1844,2064,1840,2068,1836,4020,1844,4020,1844,21824,92,12,588,

616,16,792,24,580,1876,2024,3840,4004,1852,2056,1860,2044,1860,2056,1852,4016,1848,4016,1844,4008,1856,2052,1852,2056,1852,2064,1848,2060,1844,4016,1852,2052,1848,2064,1844,2064,1852,2056,1848,4016,1848,2060,1844,2060,1844,4024,1844,2064,1840,2068,1844,4020,1840,2068,1844,2060,1844,2060,1848,2068,1844,4020,1836,2068,1844,4016,1840,2072,1844,4020,1844,4020,1836,4020,1844,4016,1848,4020,1836,2068,1848,2060,1844,2068,1836,4020,1848,4020,1840,4020,1840,4020,1844,2076,1828,4024,1840,4020,1844,4020,1844,4024,1836,2076,1836,4012,1844,4024,1840,2072,1832,4024,1844,4024,1840,2060,1844,4020,1844,4024,1840,4020,1836,4020,1848,2064,1836,4020,1848,2068,1836,4028,1836,2072,1840,4012,1844,4020,1844,4020,1848,4016,1836,2072,1844,2056,1848,4020,1844,4020,1844,22052,36,1316,12,

556,36,728,16,708,1864,2044,3820,4000,1856,2064,1852,2052,1852,2060,1844,4024,1840,4024,1844,4020,1832,2072,1844,4016,1840,2076,1836,4036,1828,2068,1836,2076,1832,2076,1832,4036,1828,2072,1836,2076,1828,2072,1840,4024,1840,4028,1832,4020,1840,2080,1832,4020,1840,2076,1828,2084,1824,2084,1828,4032,1832,2072,1832,2088,1824,2076,1828,2080,1828,4028,1832,4036,1828,4028,1836,4032,1832,4032,1828,2072,1836,2084,1828,2072,1832,4032,1832,2072,1832,4060,1800,2084,1832,4036,1816,4036,1828,4032,1832,2084,1832,4028,1824,4032,1832,4032,1832,2080,1832,2072,1832,2084,1828,4036,1820,2084,1828,4036,1828,4032,1828,4032,1828,2076,1832,4032,1836,4032,1828,4028,1828,4036,1828,2080,1832,2072,1832,4032,1832,2080,1832,4032,1824,4036,1824,2088,1824,2080,1824,4040,1824,2668,20,12,924,

140,1876,680,48,1312,1868,2032,3824,4000,1868,2044,1860,2056,1848,2072,1844,4008,1848,4016,1848,4024,1836,2064,1844,2080,1824,2076,1836,4024,1840,2068,1840,2064,1840,2080,1828,4028,1836,2072,1836,2068,1836,2080,1832,4032,1832,4028,1828,2080,1836,2076,1824,4040,1828,2076,1824,2084,1828,2076,1832,4032,1832,2088,1816,2088,1824,2072,1828,2088,1828,4028,1832,4036,1820,4044,1824,4036,1828,4028,1836,2072,1828,2092,1820,2084,1824,4036,1828,4028,1832,4032,1836,2076,1824,4036,1828,4032,1836,4036,1824,2080,1824,4036,1828,4032,1832,4028,1828,2096,1816,2076,1832,4032,1828,4044,1820,2084,1824,4032,1828,4036,1828,4052,1808,2092,1812,4032,1832,4048,1816,4032,1832,4052,1812,2076,1828,2080,1832,4036,1828,2076,1828,4036,1828,4040,1824,2076,1828,2088,1816,4036,1828,8,1412,16,16,

136,1872,544,32,1464,1868,2044,3824,3992,1864,2060,1852,2060,1844,2064,1844,4024,1840,4020,1844,4016,1840,2072,1840,2060,1848,2072,1828,4028,1840,2072,1836,2072,1836,2072,1832,4028,1836,2076,1836,2068,1836,2088,1816,4028,1836,4040,1824,2072,1836,2080,1832,4028,1832,2084,1820,2084,1820,2084,1832,4028,1836,2076,1828,2088,1824,2076,1832,2080,1820,4028,1836,4028,1836,4036,1828,4032,1832,4032,1824,2088,1824,2072,1832,2092,1820,4040,1824,4032,1824,4036,1828,2080,1832,4028,1828,4032,1832,4036,1824,2088,1824,4036,1824,4036,1828,4028,1836,2080,1824,2084,1828,4032,1828,4036,1828,2076,1828,4036,1828,4036,1828,4040,1824,2072,1832,4040,1824,4036,1828,4028,1828,4032,1832,2092,1820,2068,1836,4032,1832,2080,1820,4040,1824,4036,1832,2076,1832,2080,1828,4040,1824,-1664,12,16668,44,


[Riva]

Maybe something like attached though the 0/1 times may be inverted.
I would think section A is to allow the receiver AGC to settle before sending the sync (message start).

[pawpro]

The waveforms attached. Riva how do you call this scheme - so that I can read up.
Many thanks!

Update:
I've extrapolated from your screen grab that the data can be represented just using the timings:
Two "pulses per bit".
1. Short pulse followed by a short pulse = 1,
2. Short pulse followed by a long pulse = 0.

Out comes (4 different captures):

Code: [Select]


1,1,1,0,0,0,1,1,1,1,0,1,1,1,1,0,1,1,0,1,1,0,1,1,1,1,0,1,0,1,0,0,0,0,0,1,1,1,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,1,0,1,0,1,0,0,0,0,1,1,0
1,1,1,0,0,0,1,1,1,1,0,1,1,1,1,0,1,1,0,1,1,0,1,1,1,1,0,1,0,1,0,0,0,0,0,1,1,1,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,1,0,1,0,1,0,0,0,0,1,1,0
1,1,1,0,0,0,1,1,1,0,1,1,1,0,1,1,1,0,0,1,1,0,1,1,1,0,1,1,1,1,0,0,0,0,0,1,1,1,0,0,0,1,0,0,0,1,0,0,0,1,1,0,0,1,0,0,0,1,0,0,0,0,1,1,0,1,0,0,1,1
1,1,1,0,0,0,1,1,1,0,1,1,1,0,1,1,1,0,0,1,1,0,1,1,1,0,1,1,1,1,0,0,0,0,0,1,1,1,0,0,0,1,0,0,0,1,0,0,0,1,1,0,0,1,0,0,0,1,0,0,0,0,1,1,0,1,0,0,1,1


[Riva]

Attached is some untested code that may decode your signal.
I would like to test it with the samples you supplied before posting but time is against me and it may be the weekend before I could do that so maybe you can try it and see.

[pawpro]

@Riva: quick feedback. That is more substantial than I would have expected! Thank you very much.

With line 104:

Code: [Select]

     
      }
      rise_Time = uTime;                                  // Store rise time
    }

changed such that:

Code: [Select]

     
      }
      rise_Time = uTime + rise_Time;                      // Store rise time
    }

It does indeed detect and confirms the previous analysis.
The change may not follow logically in what we are doing but it follows your use of uTime. After getting a bit further I will rewrite this bit. For now I'll let the data flow.

Code: [Select]

We have at least 2 consecutive matching reads
11100011110100101001101110011010000111000010110101100100011001001111011
We have at least 2 consecutive matching reads
11100011110100101001101110011010000111000010110101100100011001001111011
We have at least 2 consecutive matching reads
11100011110100101001101110011110000111000010110101100100011000001011011


[robwlakes]

Just for interest sake I have written (ahem, adapted an older program) Python program to draw the timing data in a previous post (04-01-2016, 16:16:12).  The high/Low representation is just a guess at the moment. But the overall patterns may be clearer with this than from the Audacity recording.

If you would like the Python program I will clean it up and post it here as well.

You appear to have nailed it already though I see :-)

Cheers, Rob

[Riva]

Thanks for pointing out my programming error pawpro, I have altered my local copy to match.
I'm amazed it (almost) worked out of the box. Maybe I should pick some lottery numbers today 

[pawpro]

I will be back on this within 24h. But today I need to integrate Audeme's Movi board that just arrived.

[pawpro]

I'm a bit lost. I don't know if I'm doing this right.
Assuming the data capturing and decoding is right (I'm starting to think it may not be).

I'm trying to convert the binary data to dec and spot the patterns. Sadly the sensor doesn't have a screen and I don't have the weather station. So I'm using other sensors to get the readouts.
This is what I have. I am breaking the stream in to 4 bit chunks (4, 8 and 12) put together and converting it to decimal looking for my temp and humidity:

Code: [Select]

21.2C, 80%
11100010101110101111101111011000000111010100010100000100001001110000110:
 0. 1110   14;
 1. 0010    2; 11100010  226;
 2. 1011   11; 00101011   43; 111000101011 3627;
 3. 1010   10; 10111010  186; 001010111010  698;
 4. 1111   15; 10101111  175; 101110101111 2991;
 5. 1011   11; 11111011  251; 101011111011 2811;
 6. 1101   13; 10111101  189; 111110111101 4029;
 7. 1000    8; 11011000  216; 101111011000 3032;
 8. 0001    1; 10000001  129; 110110000001 3457;
 9. 1101   13; 00011101   29; 100000011101 2077;
10. 0100    4; 11010100  212; 000111010100  468;
11. 0101    5; 01000101   69; 110101000101 3397;
12. 0000    0; 01010000   80; 010001010000 1104;
13. 0100    4; 00000100    4; 010100000100 1284;
14. 0010    2; 01000010   66; 000001000010   66;
15. 0111    7; 00100111   39; 010000100111 1063;
16. 0000    0; 01110000  112; 001001110000  624;

11100010101110101111101111010100000111010100010100000100001010110101010:
 0. 1110   14;
 1. 0010    2; 11100010  226;
 2. 1011   11; 00101011   43; 111000101011 3627;
 3. 1010   10; 10111010  186; 001010111010  698;
 4. 1111   15; 10101111  175; 101110101111 2991;
 5. 1011   11; 11111011  251; 101011111011 2811;
 6. 1101   13; 10111101  189; 111110111101 4029;
 7. 0100    4; 11010100  212; 101111010100 3028;
 8. 0001    1; 01000001   65; 110101000001 3393;
 9. 1101   13; 00011101   29; 010000011101 1053;
10. 0100    4; 11010100  212; 000111010100  468;
11. 0101    5; 01000101   69; 110101000101 3397;
12. 0000    0; 01010000   80; 010001010000 1104;
13. 0100    4; 00000100    4; 010100000100 1284;
14. 0010    2; 01000010   66; 000001000010   66;
15. 1011   11; 00101011   43; 010000101011 1067;
16. 0101    5; 10110101  181; 001010110101  693;

38.5C, 20%
11100011101100010011100101011110000111000100111011000110101000001010011:
 0. 1110   14;
 1. 0011    3; 11100011  227;
 2. 1011   11; 00111011   59; 111000111011 3643;
 3. 0001    1; 10110001  177; 001110110001  945;
 4. 0011    3; 00010011   19; 101100010011 2835;
 5. 1001    9; 00111001   57; 000100111001  313;
 6. 0101    5; 10010101  149; 001110010101  917;
 7. 1110   14; 01011110   94; 100101011110 2398;
 8. 0001    1; 11100001  225; 010111100001 1505;
 9. 1100   12; 00011100   28; 111000011100 3612;
10. 0100    4; 11000100  196; 000111000100  452;
11. 1110   14; 01001110   78; 110001001110 3150;
12. 1100   12; 11101100  236; 010011101100 1260;
13. 0110    6; 11000110  198; 111011000110 3782;
14. 1010   10; 01101010  106; 110001101010 3178;
15. 0000    0; 10100000  160; 011010100000 1696;
16. 1010   10; 00001010   10; 101000001010 2570;

11100010101100010011100011111000000111010100111011000111000001100111111:
 0. 1110   14;
 1. 0010    2; 11100010  226;
 2. 1011   11; 00101011   43; 111000101011 3627;
 3. 0001    1; 10110001  177; 001010110001  689;
 4. 0011    3; 00010011   19; 101100010011 2835;
 5. 1000    8; 00111000   56; 000100111000  312;
 6. 1111   15; 10001111  143; 001110001111  911;
 7. 1000    8; 11111000  248; 100011111000 2296;
 8. 0001    1; 10000001  129; 111110000001 3969;
 9. 1101   13; 00011101   29; 100000011101 2077;
10. 0100    4; 11010100  212; 000111010100  468;
11. 1110   14; 01001110   78; 110101001110 3406;
12. 1100   12; 11101100  236; 010011101100 1260;
13. 0111    7; 11000111  199; 111011000111 3783;
14. 0000    0; 01110000  112; 110001110000 3184;
15. 0110    6; 00000110    6; 011100000110 1798;
16. 0111    7; 01100111  103; 000001100111  103;

-6.4C, 80%
11100010101010111111111101001100000111010101010000000000101100100001010:
 0. 1110   14;
 1. 0010    2; 11100010  226;
 2. 1010   10; 00101010   42; 111000101010 3626;
 3. 1011   11; 10101011  171; 001010101011  683;
 4. 1111   15; 10111111  191; 101010111111 2751;
 5. 1111   15; 11111111  255; 101111111111 3071;
 6. 0100    4; 11110100  244; 111111110100 4084;
 7. 1100   12; 01001100   76; 111101001100 3916;
 8. 0001    1; 11000001  193; 010011000001 1217;
 9. 1101   13; 00011101   29; 110000011101 3101;
10. 0101    5; 11010101  213; 000111010101  469;
11. 0100    4; 01010100   84; 110101010100 3412;
12. 0000    0; 01000000   64; 010101000000 1344;
13. 0000    0; 00000000    0; 010000000000 1024;
14. 1011   11; 00001011   11; 000000001011   11;
15. 0010    2; 10110010  178; 000010110010  178;
16. 0001    1; 00100001   33; 101100100001 2849;


Any ideas?

@Riva: I'm attaching further "improved" version of your sketch. If you can look pass the styling changes I "needed" there were couple of "serious" issues one with delay(100) being executed every loop() invocation, sync pulse detection was triggered when the pulse was longer than but not "smaller than" so was triggered very often (clearing the previous data - also changed). I've broken the debug LED a bit. 

[jremington]

I suggest to solve the temperature encoding first. Collect data at a bunch of different temperatures, trying to measure those as accurately as possible, and look for a pattern. It is very helpful to have data for temperatures above and below freezing, and to have temperatures separated by one or more single digit changes, like 20.0, 20.1 and 20.2 C.

I've decoded a number of sensors and there are two common approaches for storing temperature: always Celcius, often in tenths of a degree C, and represented by either BCD digits or binary (10-12 bits). Sometimes there is an offset so that the number is positive (e.g. 50, or 500).

So, I would look for a bit field that shows changes consistent with increasing temperature, with one of those two possibilities in mind.

This encoding is a real departure from Oregon Scientific's previous data encoding schemes (which used BCD digits), so it may or may not be very helpful to consider them.

Also, keep in mind that most of these sensors are accurate to around +/- 1 degree C, so give some leeway when comparing your own measurements.

If you like, post a table of complete messages, along with the estimated temperature, and others will be happy to help solve it. What you posted above, with the guessed breakdowns, is confusing and has too few data points for me to make sense of.

[pawpro]

@jremington thanks for your comments! I will have to set up a more controlled test. I thought they were a fairly recent model but it seems they not. So perhaps this is the old way for OS or maybe as someone suggested it's been done by another manufacturer.

Do you think the binary data we capture is correct?
Does anyone have any doubts?

Here is what you asked for. I will try to gather better data soon.
Only unique entries for past couple of hours. On a heater next to another sensor which reports 39.9C/20%

Code: [Select]


11100011101100010011100101110000000111000100111011000110100011110110111
11100011101100010011100101011110000111000100111011000110101000001010011
11100010101100010011100011111000000111010100111011000111000001100111111
11100010101100010011100011110110000111010100111011000111000010001000010
11100010101100010011100011110100000111010100111011000111000010101010011
11100010101100010011100011110010000111010100111011000111000011001100010
11100010101100010011100011110000000111010100111011000111000011101110011
11100010101100010011100011101110000111010100111011000111000100010001110
11100010101100010011100011101100000111010100111011000111000100110011111
11100010101100010011100011011110000111010100111011000111001000000011111
11100010101100010011100011011100000111010100111011000111001000100100000
11100010101100010011100011011000000111010100111011000111001001101000000
11100010101100010011100011010100000111010100111011000111001010101100100
11100010101100010011100011010010000111010100111011000111001011001110011
11100010101100010011100011010000000111010100111011000111001011110000100
11100010101100010011100011001110000111010100111011000111001100010011111
11100010101100010011100011010110000111010100111011000111001010001010011