What if you do something like this:
display.setCursor(0,40);
display.setTextSize(3);
display.print(" "); // print blank spaces to erase previous value?
display.print(encoderPos, DEC);
Also, looked at the library, the display.display(); should be called last, after you draw/print things, because whe you print something it just adds to a buffer, and the display() function is what copy that buffer to the display itself.
But as you call it at the top of the loop, it's like if it was at the end, no difference, just confusing.