Did you remember to connect the ground from the external 3.3v supply to the GND pin on the arduino?
If so then, I will need to do further tests when my pro micro comes in.
What about when it isn't being used with the joystick library, does it work on its own correctly? If it does then maybe the two libraries are conflicting with each other.