Hello Palliser,
I have a problem on Arduino Mini to, because I replaced with faster DUE. On both the same error whit the same code. If the encoder quickly turn signals are lost in the step signal. I surveyed with an oscilloscope. Even when I used the code for direct manipulation port (MINI) there was no improvement.
To the Arduino due I used encoder with 3.3V signal, because it can destroy due if use 5V signal.
I tried all the possibilities but only one I have left to recompile the code in 32bit operation.
Work 8bit is slow, so the only option that works in 32bit.
The program only works if you slowly turn the encoder. If you are on the encoder fast scrolling code crease signal.
View: Arduino Encoder to Stepper Motor - YouTube