Let's say you start listening in the middle of the data stream. The "wrong" 'C' arrives and you then read 24 bytes and don't get an 'F'. So you read again. Meanwhile the other end has started sending again (quite likely since you might have only received 10 bytes before the "real" 'F'). Since the other end is retransmitting you get the wrong 'C' again, and so on. Nothing would really break you out of this loop.
A simple solution would be to send in ASCII and not binary, take the slight performance hit, and then put non-numeric characters at each end, eg.
If the numbers are small, something like "77," is actually less bytes than a long, anyway.