Potentiometer Readings Off

Can you explain this please?
There is nothing to suggest negative numbers in your code.

Did you mean

int brightness = analogValue / 4;

?