I think the output (communication from the computer to the Arduino) is a problem. Processing, for example, has a standard library for doing serial communication, but I don't see anything for calling low-level HID output functions. I imagine other languages have similar issues. As a general purpose device, it's important for the Arduino to be able to flexibly communicate with software.
libhid is an easy way to get full USB HID support on Mac OS X and GNU/Linux (not Windows, unfortunately). I've started a Pd wrapper for it, there is also a python wrapper for it. C, C++, Obj-C, and C# all have full-fledged HID, AFAIK. This stuff is not quite there yet, but it's getting there. That's what I think it would be nice to replace the FTDI with AVR-USB. It could still be USB-serial now, but in the future, it would just be a firmware upgrade to get it to be USB HID.
I'm concerned about the input side, too... sure, the Arduino could pretend to be a mouse or joystick, but what happens if you need to communicate data that can't be squeezed into an X and Y coordinate (or whatever data format is predefined for those HID devices)?
Maybe this would make more sense as a (slightly) different device? One that comes preprogrammed with firmata, say, and so only needs to send specific types of information would could fit in a HID protocol?
I guess I don't see what would not fit into HID packages. These would all fit into a single type: digital I/O, analog In, PWM out, pulse I/O. Then for something like shiftOut, you could use three data types which are sent in one report. That's a common thing to do in HID.
FYI, phidgets all use HID, AFAIK. Plus there is a free library for it, which is one approach to fully supporting I/O:
Oh, and the reason behind the proposed shield having its own ATmega is because that one could be running at 12 MHz. Plus, that would take the burden of having the AVR-USB implementation off of the main ATmega. There could be some simple protocol for the main chip to send a keystroke, say, to the usb ATmega to send to the computer.
Right, I guess those are the same reasons I have proposed replacing the FTDI with an ATmega. A shield could be handy too.