I see you are using the Keyes_SJoys PS2 Joystick Game Controller Breakout Module For Arduino from ebay. If yours are anything like mine, then they are of no use for fine control. Attaching this to my multimeter (between GND and VRx or VRy) gives very non-linear results:
At rest in the middle gives approximately 3.5kohm. Pressed to one extreme gives a value of 100-130 ohms (depending on which pot I am reading). This value does not appreciably change as you relax the stick towards rest for a good third of its travel. Then the resistance actually goes DOWN to between 50-100ohms and then rapidly climbs back up to 3.5kohm in the last third of its travel. It is also not reproducable in the change region. i.e. if I reverse direction to a previous position, I do not get very reproducable results.
Going the other direction is much the same. At its extreme, I get about 4.6kohms, then UP to 5kohm as I relax it towards the rest (middle point) and then down again to 3.5 kohm at its central rest position.
The results are not reproducable, do not change appreciably for a good third of travel at the extremes and in the middle are completely counter-intuitive (reversing resistance direction for some of its travel). At best, these devices will only be usable as a detection for extreme up, left, right, down and middle. Also, they can be used for 45 degree angles of up and left, bottom and right, etc. The advertised momentary button (SW) also only works in the rest position (middle). I would not use them for detecting any form of graduated change in direction.
Even for the detection of the extremes, there is a potential for false detection if a user simply bumps the stick slightly into the resistance reversing region while making a change in another intended direction. They are truly not great products, but then you often get what you pay for ...
I bought these as a very cheap system of getting a simple input for menu selection and basic navigation because I can still use them with just 3 pins on my AVR as low pincount is an issue in my design, so they kind of work. Effectively, they permit a total of 8 detectable positions plus the central rest position together with the momentary button, which is not too shabby for the use of just 3 pins.
Unfortunately, the quirks of these devices will cost me a lot of time and code in programming to detect whether we are truly at an extreme or whether we are just passing fleetingly through the resistance reversing direction. I think I might just require multiple consitent ADC (with a given tolerance) as an indicator that we are not fleetingly in the reversing area.
I am not unique in having these issues with these types of devices. See the comments on the sparkfun website (https://www.sparkfun.com/products/9032
) for their similar item.
Also note that the momentary button has no resistance associated with it. This might be an issue for you depending on what you are trying to drive with it as use of any appreciable amount of current through this switch is reported to burn out these joysticks.
Given the other issues you have had with your code, these issues with this device are likely to make your project very difficult for you.