The 5v was the only one that gave me the correct result (1023) and I suppose that a smaller potential difference would give me 1023
No the 5V line is the voltage reference voltage so suppose it is low in terms of absolute voltage say 4.8V measuring this with a 4.8V reference will still give you 1023.
What you need to do is to have and external stable accurate Vref. Then the figures will come out right. When you input a voltage equal to this Vref you will measure 1023.