Byte compression for non-repeated bytes

I'm doing byte transfer from arduino uno to raspberry pi via spi. i have to transfer around 50KB per second which can be increased later. Now im looking for data compression coding to reduce cpu usage in raspberry pi(now im running @4MHz). I found run length encoding for repeated bytes in a message. But my sensor data in arduino may not repeated bytes in a message. so i'm looking for an idea or code to implement it. Thanks Venkatesh

i have to transfer around 50KB per second which can be increased later.

The best compression technique is to not transmit values that have not changed. If there is a lot of churn in the data, no compression technique works well.

50kbyte or 50kBit ?

50KBytes.

Can you give some example data as the compression strongly depends on the type data.

Now i'm establishing communication base for my project. So the sensor's are not finalized, as far now bytes may be random for practice. Now im keeping 25-50 bytes in a message and trying to send as much as possible messages. If bytes are compressible i can send more messages. For example, if the bytes are 0xA5,0xCD,0x08,0x34,0xC3,0x56,0x39,0x77,0x98,0xDE,....... is any possible way to compress it message length may be 25-50 bytes. Difficulty in spi of arduino is we can able to send only one byte at a time in a transfer. If compression is possible it may overcome the issue. By means of compressing i can achieve more transfer rate and cpu usage in rpi will be reduced.

By means of compressing i can achieve more transfer rate and cpu usage in rpi will be reduced.

If (and I'm not suggesting it is possible) you are able to compress the data, the cpu usage on the Pi will be INCREASED because it will need to decompress the data.

So the sensor’s are not finalized, as far now bytes may be random for practice.

random data cannot be compressed by definition.

to be compressible the data should be either have repeating patterns (not necessary equal length) or be predictable by some formula (but this is difficult and you give up the real time aspect).
Another way of compression is sending the delta between 2 bytes but this only works if there are no big jumps in the original data.
Dictionary compression like Huffman or G4 is based upon the known frequency of certain numbers. There exist adaptive Huffman’s but these take even more CPU.

What kind of messages are you sending? If its something like text to be displayed on a display, you can do things like send over a string of numbers from 0 to 255, each number represents the font pattern to be displayed, the receiver pulls the font pattern out of flash and writes it to the display. If the font is 5 columns wide that represents with 1 blank column between letters, that represents a 6:1 compression. Other kinds of simple swapping may also be possible depending on your data.