I don't see any problem in the code. If you change:
#define XPOT_PIN A0
#define YPOT_PIN A1
to
#define XPOT_PIN A1
#define YPOT_PIN A0
Does the problem move to the X axis? If so, that would indicate a hardware problem with the pot on A1