Virtual Reality Input Device - Is it possible with Arduino ?

Hey everyone,

So I’ve done some looking around and I’m new enough to Arduino, having only a year of basic knowledge behind me. I have done some research but cant quite seem to find my answer. I am developing an input device, or a game controller, similar to the Wii nunchuck. And to save myself some money and time, planning is everything. So my question to you is, is it possible to develop a game controller using arduino along with extra components, and use it to play an EXISTING game (PS1 emulated games or other basic types) for demonstration? I have looked online and found a few project people have come up with but all they show is numbers on a serial monitor, and no real application use. I’m unaware if there are problems in the way such as drivers, compatibility etc.

I have gone and bought a Game Shield for the Arduino, along with a couple NRF24L01+ modules to implement wireless abilities, as it is a bonus for the final product. Theyre also perfect as they slot right into the game shield.

What do you guys think? I’m confident about making and assembling this project, but I’m unsure about its final use to actually PLAY a game.

If you want something like a Wii nunchuck (maybe with extra buttons) then start with a Wii nunchuck. The hard work is already done for you. Emulating something that complex from scratch, with limited knowledge, is just not going to work in the time you have available.

Yes this is possible but it depends on the interface which the gaming PC or consol presents to the outside world. Your Arduino has to talk the same language in the same way. Many games use a simple USB serial interface and that is why you see so many projects just showing numbers on a serial monitor, because it is showing you the raw data input. Connect those to the appropriate system and the game will play.

MorganS: If you want something like a Wii nunchuck (maybe with extra buttons) then start with a Wii nunchuck.

You have a good point. I was looking at taking apart a Wii controller on nunchuck for the final (pretty) prototype for aesthetics as an Arduino won fit into the size of the controller I have in mind. I thought I can have a developed, functioning prototype (messy) to show more work. Perhaps there's no need for the breadboard though you may be right. Thank you.

Grumpy_Mike: Yes this is possible but it depends on the interface which the gaming PC or consol presents to the outside world. Your Arduino has to talk the same language in the same way...... Connect those to the appropriate system and the game will play.

So once the language and connection (Bluetooth or USB) work with the game and PC it should work fine?

So once the language and connection (Bluetooth or USB) work with the game and PC it should work fine?

No the language has nothing to do with it. As long as the interface passes the numbers that the game requires it will work. What language is used to generate those numbers is irrelevant.

Thank you. I've another question for anyone who can help, as for the Wii Nunchuck. The controller has limited inputs, just an analog stick, a bumper button and a trigger. Would it be possible to increase the functionality of the controller by adding extra buttons, and adding the to the circuitry or is that just out of a developers reach? Any suggestions?

Ryan_Williams: Thank you. I've another question for anyone who can help, as for the Wii Nunchuck. The controller has limited inputs, just an analog stick, a bumper button and a trigger. Would it be possible to increase the functionality of the controller by adding extra buttons, and adding the to the circuitry or is that just out of a developers reach? Any suggestions?

I'm trying to figure out what you mean by the above.

You can add anything you want to a Nunchuck but the controller has to generate data in the correct protocol in order for the game to make use of the controller's data.

BTW, a Wii Nunchuck also has a 3-axis accelerometer which can be used as a second joystick. I've used a Nunchuck to fly a RC helicopter by using the accelerometer to control the yaw and throttle.

If you wanted to use a Nunchuck to control a Playstation 2 console, you could use the accelerometer to generate the data for the missing joystick.

Whatever buttons or triggers you add, you'll need to generate the appropriate data signal expected by the game console. You won't be able to add buttons or triggers if there isn't a corresponding data field in which to send the new button or trigger's state.

Ryan_Williams: So once the language and connection (Bluetooth or USB) work with the game and PC it should work fine?

I imagine by "language" you meant "protocol"?

Even once you can generate a signal in the appropriate protocol, you need to make sure the controller is sending data in way the console is expecting.

DuaneDegn:
I’m trying to figure out what you mean by the above.

Sorry for the confusion. The Nunchuck has as mentioned the joystick and two bumper buttons. I was looking to add extra buttons such as an A B X Y like you would find on an Xbox controller. But I was unsure about the feasibility. You answered my question well enough though so I have a rough understanding. So even if i managed to add more buttons to the controller they would have to send a signal the controller/game can understand? If its too much work I may leave out the functionality of the buttons and have them there simple for aesthetics to show the final product.

I have attached some sketches of what I have in mind.

So even if i managed to add more buttons to the controller they would have to send a signal the controller/game can understand?

Yes. The game has to understand what you send it, that is it needs already to be prepared to do the thing that you want your button to activate.

Ryan_Williams:
Sorry for the confusion. The Nunchuck has as mentioned the joystick and two bumper buttons. I was looking to add extra buttons such as an A B X Y like you would find on an Xbox controller. But I was unsure about the feasibility. You answered my question well enough though so I have a rough understanding. So even if i managed to add more buttons to the controller they would have to send a signal the controller/game can understand? If its too much work I may leave out the functionality of the buttons and have them there simple for aesthetics to show the final product.

I have attached some sketches of what I have in mind.

Here are the attached images so others don’t have to download them.

6e1f785ffa8782d9487264de420987ddd9c84a57.jpg

0f24a687d6239a49f86002fcdd56d7ee37343b2c.jpg

How to insert uploaded images.

What do you want to control with your input device?

Even with a Nunchuck, it’s a bit awkward to press the “C” and “Z” buttons while also holding a position with the joystick. I don’t think the three buttons below the analog input will be very useful if the person wants to maintain a position using the analog input. It looks like one could use the analog input “XOR” (exclusive or) use the buttons. It doesn’t look like they could easily be used at the same time.

DuaneDegn:
I don’t think the three buttons below the analog input will be very useful if the person wants to maintain a position using the analog input. It looks like one could use the analog input “XOR” (exclusive or) use the buttons. It doesn’t look like they could easily be used at the same time.

That’s a fair and true point. Ergonomics and interaction plays a huge role in the controller. Having done research into existing controllers for VR games like the STEM SIXENSE or Oculus Touch, they all have a similar layout. This is why with the pairing of two devices the left hand controller can be used to control the walking and position where the right hand can be used for interactions. The layout is even the same for console games.

None the less having experienced VR games and applications, the experience can be often slow paced, leaving you enough time to take your thumb off the analog stick to press any of the buttons below the analog.

But perhaps you’re right and you raise a good point, the buttons could be placed beside the analog to make for better user interaction, or in another place rather than below, which does take more time and work to do. Having played on PC games with a keyboard the past year and a half, you can imaging how poor the interaction with a keyboard can be, sprawling my hand to reach far off buttons.

Hey guys, time to revive this discussion, I need some more help. I have my design pretty much finished and I am in the process of making the gamepad / joystick. I can't fully understand or figure out how I can get my arduino uno or nano to communicate with my laptop/PC and run any games. Does anyone have any ideas how to interface my arduino with a game, the likes of Counter Strike Global Offensive for example (although game shouldn't matter).

although game shouldn't matter

Yes the game will matter. It depends on how the game is looking for the input as we said before. Do you know this?

To be honest I wouldnt have a clue. I used UnoJoy which flashed the bootloader to recognize it as a PS4 joystick and it ran fine with some tweaking to the key bindings, but when I tried to make it wireless it went haywire. I think using the NRF24L01 was using the pins needed for the game. Was just wondering if there was any other way and how I'd go about doing it?

I used UnoJoy which flashed the bootloader to recognize it as a PS4 joystick and it ran fine with some tweaking to the key bindings

OK that looks promising is it a PS4 joystick you want to emulate? You will need to know what joystick keys to emulate.

The other solution is to use an Leonardo or a Micro to send keyboard strokes to your PC, if that is what your game wants.

but when I tried to make it wireless it went haywire.

No fundamental technical reason why that should happen.

I think using the NRF24L01 was using the pins needed for the game.

That is something you should know not just think. It is easy enough to find out, just look what your code was doing.

When you have UnoJoy installed I assume you can't program the Arduino normally and you will have to use some sort of programmer. Another Arduino Uno can be made to do this.

Yeah Emulating it as a PS4 joystick would work as my controller design has the same number of buttons. The transceiver is connected to pins the emulator would use as a button, but I had those lines // out. Its something I could work around I'm sure.

Using a micro or Leonardo could be interesting. Since PC games run fine with keyboard strokes. Would I be able to use a Micro (easier as it'd fit into my design), have buttons connected, when pressed they will emulate WASD and other buttons? Ideally it'd be wireless.

For wireless I could connect buttons that are when pressed send a signal to a receiver HUB at the PC, the Leo or Micro, and execute the keystroke?

Yes with a micro that is what you can do. Not even single strokes but key combinations and rapid sequences can be sent to the PC just as if you had pressed the keys. If the micro is too big then look at the beetle, it has the same processor but not as many pins are brought out, but you can get round that in many ways. Also they do a Bluetooth version of the beetle.

I feel this may help communicate my design to anyone a little better., along with my finding to help anyone looking to do something similar.

I researched a little and I have two options now.

Option A
Use two Arduino Micros as there are two different controllers. I found a useful video here showing how to use the micro to simulate keybindings and make a retro arcade controller. However I would need some extra parts to make this wireless and integrate an internal battery with a recharge adapter module.

Option B
This option is more straight forward and involves using the Adafruit Bluefruit EZ-Key which acts as a wireless HID developer board. This is more applicable as it was built specifically for this reason. There is another neat little video here which shows how to make a wireless game controller. Again for my design I believe I will need two of these, one in each controller to connect to the PC. Along with the additional extras to power the board.

Option A is a cheaper option, with the micros at around €7 a piece, but it is more time consuming and more difficult to work with as it isnt wireless. Option B is more expensive with the EZ-Key at €25 a piece, but it is a lot easier to work with it seems with the built in bluetooth. However there is also little information on the code or bindiung for the keys on the EZ-Key, perhaps its all setup in the game settings.

Thanks! I had not seen those blue fruit things before. I will definitely buy a few to play with. It's not clear how flexible they are to emulate a joystick for your project.