Hi.
We use arduinos in one of our courses on digital art. Our students are artists, not programmers, so I try to keep things as simple as possible for them. One of the biggest areas of difficulty/frustration we encounter is interfacing sensor-laden arduinos upstream to PCs as the studenes invariably want to go well beyond what an arduino is intended to do on its own (play video fragments in response to inputs, mainly). Usually this means controlling a media player or other similar software. One of our teacher's solutions was to add PC-side programming (scala in his case) on top of Arduino programming to allow this, which basically overloaded the students and trippled the teacher's own workload as he ended up having to heavily debug most of the code for them on a continuous basis. My own solution is "you can't (easily) do that with an Arduino" which I am less than happy with too.
The ability for the Uno/Mega256 to be re-configured as a USB HID device looked like a good solution, but in the end the mucking about with the controller firmware is too painful, especially since you appear to have to reload serial/USB function every time you want to reload/test your sketch! The boards get used by multiple students each across the week too, so we don't like to let changes that can't be reversed by just plugging them in and loading a new sketch.
What would be ideal (assuming it is possible) would be a function for the interface chip that can register it as a USB composite device of Serial (for programming and serial comms), keyboard, joystick (and mouse, midi, as reality permits) so that any/all can function at once (or even just serial+other would be fine if you could switch the protocol of 'other' from the sketch).
Some sort of command such as setUSBHID(KEYBOARD|MOUSE|JOYSTICK|MIDI,TRUE|FALSE) to activate/deactivate the various HID comms modes in the sketch initialiser. And associated commands to write data out to the appropriate interface, of course: writeKeyChar(char keyValue), writeKey(char keyValue, boolean press/release), writeScanCode(char scanRow, char scanCol), writeJSButton(integer button#,boolean value), writeAnalogStick(integer channel, integer value), writeMouseButton(integer button#,boolean value), writeMouseMotion(integer x,integer y), writeMouseAbsolute(integer x,integer y), etc. For example.
Does any of this sound doable?
Regards.