When i use the TX RX for optical rotary encoder to use as mouse cursor X/Y, everything goes smooth.
However, with the exact same code on any of the other ports it barely moves and is insanely jittery.
Is my arduino micro fried?
What is this "exact same code " of which you speak?
How have you wired this up?
You might want to look at this How to get the best out of this forum before you proceed any further.
When you change tx/rx pins, be sure to update the tx/rx pins in your code.