Joystick for Xbox Adaptive Controller

Hello all, hope everyone is doing well. I've been working on a project with the Xbox Adaptive Controller and an ESP32. Here is a little background.

I have a physical disability that effects my muscles (cerebral palsy) and I'm not able to use a regular Xbox controller. I bought an Xbox Adaptive Controller, hooked up 19 relays to it and an ESP32, wrote a python script that runs on my communication device (controlled by my eyes) which sends messages to the ESP32 via bluetooth telling which relays to trip/untrip. This works great.

The only draw back to this has been, with the thumb, I only have options of on or off. Meaning I can only move them either 0% or 100%, no middle ground. The adaptive controller has two USB ports where you can plug in joysticks and they become the thumb sticks. So I had the idea of using a Leopard with a bluetooth module and the joystick library to give me analog joystick control and wrote the following code.

#include <SoftwareSerial.h>
#include "Joystick.h"

SoftwareSerial mySerial(10, 11);  // RX, TX
String message = "";              // Buffer to store incoming message
String action;
int button;
int xValue, yValue;
bool firstRun = false;

void setup() {
  Joystick.begin();
  mySerial.begin(9600);
  Serial.begin(9600);
}

void loop() {
  if(!firstRun){
  Joystick.setZAxis(0);
  Joystick.setXAxis(0);
  Joystick.setYAxis(0);
  firstRun=true;
  }
  // Check if data is available in the serial buffer
  while (mySerial.available()) {
    char c = (char)mySerial.read();  // Read one character

    if (c == '\n') {
      // When newline is encountered, process the full message
      processMessage(message);
      message = "";  // Clear the buffer for the next message
    } else {
      // Accumulate the characters into the buffer
      message += c;
    }
  }
}

void processMessage(String msg) {
  if (msg.length() > 0) {
    // The first character is the action (e.g., 'p' or 'r')
    action = msg.charAt(0);
    
    // The rest of the message (from index 1 onwards) is the button number
    String tmpButton = msg.substring(1);
    
    
    if (action == "p" || action == "r") {
      button = tmpButton.toInt();
      // Handle digitalWrite for buttons 0-8
      if (button >= 0 && button < 9) {
        if (action == "p") {
          digitalWrite(button, HIGH);
        } else if (action == "r") {
          digitalWrite(button, LOW);
        }
      }
      // Handle Joystick button actions for button >= 10
      else if (button >= 10) {
        button -= 10;
        if (action == "p") {
          Joystick.pressButton(button);
        } else if (action == "r") {
          Joystick.releaseButton(button);
        }
      }
    }
    else if(action == "l") {
      int commaIndex = msg.indexOf(',');
      if (commaIndex != -1) {
        // Extract the X value (substring after 'l' and before the comma)
        String xStr = msg.substring(1, commaIndex);
        // Extract the Y value (substring after the comma)
        String yStr = msg.substring(commaIndex + 1);

        // Convert the strings to integers
        xValue = xStr.toInt();
        yValue = yStr.toInt();

        // Set the X and Y axes of the joystick
        Joystick.setXAxis(xValue);
        Joystick.setYAxis(yValue);
      }
    }
  }
}

The idea is, I'm still going to have a couple of relays for the D-pad since you apparently can't have D-pad commands go through the USB ports, but you can send commands such as Button 1, Button 2, Button 3, etc. through the USB. The problem I'm having is, no matter what I send to the leopard via bluetooth, the left thumb stick always goes to the left and stays there, even if I sent "l0,0". The problem does not exist when I plug the leopard into my computer, only with the adaptive controller.

I know this is kind of difficult to follow but I'm hoping someone might be able to help me figure out what I'm doing wrong?

The USB joystick HID report descriptor can specify the axes values be signed or unsigned integers.

If the axes are unsigned, (0,0) is the upper left corner. (127,127) is the center.

If the axes are signed, (0,0) is center.

Which joystick library does the code use?

Very cool project. I have been experimenting with a USB joystick for ESP32-S3 that controls both XAC joysticks, the dpad, and 12 buttons. This is works because Microsoft upgraded the XAC firmware to support this advanced functionality. Maybe this eliminates the need for relays to control the dpad?

Hello, thank you for the response. I'm using the library from Matthew Heironimus. The stranger thing is, even if I send "p13", the left thumb stick still goes left all the way.

Yeah after I posted this question, I did read that they updated the XAC which allows for D-pad commands to go through the USB port via the hat so once I figure this problem out, I am going to take the relays out of the equation.

Have you gotten yours to work with the ESP32-S3?

My experiment with ESP32-S3 is not working but there is hope because someone mentioned on /r/disabledgamers that a Logitech gamepad in DInput mode can control the XAC left and right joysticks, dpad, and 12 buttons. I may switch to Leonardo/Pro Micro because compiling for ESP32-S3 is SLOW. I stopped working on it but will try to pick up again.

Try initializing the joystick like this. I do not have a bluetooth module so cannot test.

void setup() {
  Joystick.setXAxisRange(-127, 127);
  Joystick.setYAxisRange(-127, 127);
  Joystick.setZAxisRange(-127, 127);
  Joystick.begin();
  Joystick.setXAxis(0);
  Joystick.setYAxis(0);
  Joystick.setZAxis(0);

It is possible the default joystick is too complex for the XAC. The default joystick has 13 axes and 32 buttons.

There is something odd about the XAC. Pressing a USB joystick button can move an XAC joystick. This behavior can be changed using the Xbox Accessory app (available on Xbox Console and Windows). The Windows app is free from the Microsoft store.

For example, if a logitech joystick is plugged in to an XAC, pressing the logitech joystick front trigger button (button 0) moves the XAC left stick.

This Microsoft video shows how to remap this to a useful function such as pressing the RB button.

@brad3260 This library emulates a DInput gamepad using a Leonardo. It is compatible with the XAC. The ESP32-S3 version is still unusable.

This has only been tested on a Arduino Leonardo board. But it should work Arduino/SparkFun Pro Micro and Arduino Micro because they all use the same microcontroller.

The gamepad has the following controls.

Control Description
X 8 bits, 0..255, left stick
Y 8 bits, 0..255, left stick
Z 8 bits, 0..255, right stick
RZ 8 bits, 0..255, right stick
Hat 8 way hat switch/direction pad
Buttons 12 buttons

Duuuuuuuuuude!!! Thank you so much!!! I am going to give this a try. My plan was to have two Leonardoes, one plugged into the left and another to plug into the right USB ports. However looks like I don't have to do that with this library. Thank you again and I'll update you with results.

1 Like

Now that I got it work with Leonardo, it was easier to get it work with Raspberry Pico with the Adafruit TinyUSB library. The Pico 2 W was recently announced so I will test when I get one. The W variant has Bluetooth and WiFi for more interfacing options.

The Adafruit TinyUSB library also supports nRF52840, SAMD21/SAMD51, etc. so there are more options which I do not have time to test.

This is the last one for ESP32-S3: GitHub - controllercustom/dinput_ESP32: Arduino ESP32-S3 DInput gamepad controller for Xbox Adaptive Controller

The ESP32-S3 dev boards with two USB ports are handy to bridge between a PC and an XAC. I may be able to get Voice Attack to work on Xbox console using this.

This is super cool and is allowing me more flexibility than using relays. It was appear I am going to need one relay though for the Xbox button, unless you have figured out a way to do that over USB? Also, from my understanding, on the XAC, the X1 and X2 "buttons" are analog so I assigned the bumpers to those. Have you figured out a way to send a value to those instead of just press?

If an analog joystick is plugged into X1 or X2 jacks, the joystick controls the XAC left or right joystick. Since dinput can control the XAC joysticks, I do not see a need to drive X1 or X2 input jacks.

On the other hand, the LT and RT input jacks are analog inputs that control the XAC left and right triggers/throttles. These are usually used in driving games to control gas and brake. dinput cannot control the XAC left and right triggers. One possibility is to connect digital potentiometers (digipots) to the LT and RT input jacks. Digipots are software controllable (I2C or SPI) variable resistors so can replace analog controls.

Are you aware of the HID remapper project? The v7 board has relays and digipots. Maybe this can do what you want. I have never tried it because a complete unit costs 99 pounds plus shipping/handling from the UK. Or buy a minimum of 5 boards from a Chinese PCB factory. Neither option appeals to me because this is a personal project. I do not need to drive every feature of the XAC.

As far as I can tell, an external joystick like dinput cannot control the Xbox button. A relay is required.