Hello Rob, thank you very much for all the input!
Don't get me wrong, efficient data encoding will be very important further down the road, but I found the Serial.print to be so slow that no matter how good compression will be, I will need a faster data transmission option.
Have a look at these numbers:
Full speed sampling to internal memory (i.e. no communication): 100 Hz
Sampling + sending raw data through SPI: 73 Hz
Sampling + sending raw data through Serial.print (baudrate 115200): 2.2 Hz
As you can see Serial.print will not be an option in this application.
It will be a real time system with several hundreds of sensors connected to the Arduino. I can't give you any exact numbers as of now but we will need several Arduinos i.e. the maximum amount of sensors an Arduino will be able to sample and transmit in real time will be used.
In the end I will need an option that is able to transmit at least 150+ KB/s, possibly >1MB/s which brings me back to the question: is there some SPI to USB hardware that I could just connect to the Arduino SPI pins and that will handle the rest?
Have a nice day my friends!
Tom