From the comments above about the Xinput library not supporting the rotary encoder, it appears that you currently do not have enough information on how to proceed. I suggest to post on gaming forums for advice, as the Arduino forum has a different focus.
It might be possible make the rotary coder "look like" some input that the library supports, but again, you will need detailed information about the expected data protocols.