Not knowing what type of outputs your encoder has. Are they TTL (5V) or are they differential? Differential outputs may not have enough voltage swing. If that is the situation then you would need a differential reciever to convert to a 5V TTL level signal for the Arduino.
I have used that code with an encoder so I know it works.