22k is way too high for an LED on 5V. You are going to have to use the bottom 1-2% of the range of the pot.
That focuses all of the heating into that one small part of the pot's resistive strip and it will be toasted, even if you somehow never accidentally hit 0%, which cooks everything else.