Rotary Encoder - learn this if you have about 400 hours of time.

First advice is often to post your code (use the # button for code tags)
Also post a schematic how things are connected as we can only guess.

Many questions pop up:
Do you use interrupts?
If so are the interrupt variables volatile?
Do you use delay or print statements in interrupts?
Does your code have deadlock or livelock or race conditions?
do you use pull up/down resistors?
etc

We just cannot tell without code / schema

A good way is to strip your code to a minimal version that still show the problematic behaviour as that is easier to fix than code that includes a complete complex project.