Now I am trying to work out the CRC in the final byte so I can add error checking to the sketch to eliminate the 3% of misreadings.
From the 1,000 + unique data captures I have selected three sets with a single bit difference - I hoped I would be able to break the code from them but failed. Time consuming attempts to match it manually using the CRC-8 examples given on WIki have also been unsuccessful.
Since I posted I have discovered an 8 year old post in a different forum regarding checksum algorithm reverse engineering. Following their method I discovered that there is a consistent relationship between the single bit changes across the sets of data I provided above and other sets I have located within my other data points. I also noticed that one of the data points in set 3 is wrong.
Thanks Jim for running reveng and posting the result.
I have now been to sourceforge.net and read the reveng manual but am still a bit unclear as to what the result means. (I also tried running reveng myself on my virtual box but could not make it work).
In relation to the ff results
check=0x2fe0, what is the check?
Is the first width and poly (16 and 0x0001) part of the query to reveng, or part of the result?
If I wanted to incorporate this detail into an error checking sketch how do I use this data?
reveng -w 16 -a 8 -s ff4577148031a5 ff457714823123 ff4577148131e6
The command states to use a 16-bit shift register, that the character width is 8 bits (default and not needed) and to search the following three strings for an appropriate algorithm. You may have to read up on how CRC computations work to figure everything out, e.g. Cyclic redundancy check - Wikipedia
Note: this works for the first 3 or so messages in every set, but not all messages. You may have some errors in the messages, as suggested by your post, or you are still missing something. The fact that the same polynomial keeps coming up is encouraging. The highest order bit of the polynomial “signature” is dropped by convention, so the actual feedback polynomial 0x0043 reported by reveng for your data means to use bits 15 and 6, 1 and 0 as the feedback terms.
There are several websites that will calculate the CRC for an arbitrary polynomial and other options, and even write the C code for you, but it may take several tries to get it right. This is because there is no standard for the bit order (LSB first or MSB first), either for the individual bytes or for the entire message (hence the “reverse” options in reveng). In addition, there are the possibilities of XORing the data with initial or final values (INIT, XOROUT). Because of this, it often doesn’t matter if “extra” constant bytes appear in the message, like FF or FD, as they get swallowed up by the INIT or XOROUT terms.
C Code generator site: https://ghsi.de/CRC/index.php
To get started with the above site, enter “10000000001000011” as your polynomial and leave out the leading FF or FD bytes of the messages. You will see that for the first three entries in each group the messages always result in the same CRC value, which could be taken care of using the “XOROUT” term.
I wondered about the “check” and wrote to Greg Cook (author of reveng) about it. Here is what he said:
The Check value is the CRC of the ASCII string “123456789” in the specified algorithm. It’s there as a check that the algorithm has been implemented correctly. This field was introduced by Ross Williams in chapter 16 of his 1993 paper “A Painless Guide to CRC Error Detection Algorithms” http://www.repairfaq.org/filipg/LINK/F_crc_v34.html#CRCV_005. As indicated there, it is a ‘non-normative’ field - the other fields are authoritative in the event of a discrepancy.
Edit: Note also that some remote sensors swap the nibbles within the bytes, so that is something to try. Good luck and let us know how you get on!
I played with your data a bit more but am not convinced that the solution has been found. It also seems unlikely that a 16-bit shift register would be used. So, I tried bit-reversing each of the message bytes and looking for a CRC-8, but that didn't work either.