but just curious as to what could be causing this.
You may have the potentiometer miswired so that it is shorting the power supply (and overloading the potentiometer) when you turn it to one end.
This is after an hour of wondering why the contrast didn't adjust at all only to find out the pin2 and pin 3 are swapped on my particular HD44780 compatible LCD (JHD204A - made in China).
I doubt it. Are you sure you haven't misinterpreted or miswired the potentiometer pins.
Don