Think about it: Serial output can affect the time it takes to complete portions of your code. I have seen this mask timing bugs like the one Nick points out above on many occasions. When the output is turned off, the bug appears.
Even if you know there are 24 chars in the packet, unless you check to see whether all 24 have arrived when you read them, you run the risk of stuffing your read buffer full of "-1" as you read past the end of what has actually arrived so far.
Could this explain the results you are seeing?
-br