potmeter returns values 0, 1, 2 & 3 instead of 0 - 1023

Have you got the grounds of the two Arduinos connected together?

If not you won't get a proper reading.
You need to connect the two grounds together so that the second arduino has a zero volt reference to measure from.

If you were trying to measure the voltage with a multimeter, you wouldn't expect to get a correct reading with only the red probe connected, would you? That is in effect what you were trying to do.