In short, the first and most important thing if using manually operated rotary encoders is - do not use interrupts.
And the second follows - you need a proper debounce routine.
Something which I am working on at present (as it happens).
And of course, we have to say.