porting midibox functionality to Arduino?

There are a bunch of things that I can do with the midibox platform that I would like to be able to do with the arduino. A lot of the boards that TK has designed (DOUT, DIN, AIN etc.) should be interfaceable to the Diecimilia platform.

64 rgb leds (by cascading three max7221 chips, 1 for each colour)
64 tactile switches (e.g. digital in (with 8 cascaded 74HC195 you only need 3 digital pins on the diecimila to interface)
32 analog in (e.g. pots) (on the midibox core you can connect 4 4051 mux to 4 analog input pins on the PIC.)
Midi i/o (via usb+serial2midi driver, i2c, or another way?)

I would also like to control some sort of graphics display if there are enough pins left. (e.g. segmented display, lcd etc.)
Also, depending on how I implement the midi i/o I would like to chain multiple diecimilas together so that they all receive and send midi on the same interface.

Is the response time on the input controls going to be too slow? A huge advantage on the midibox system is that code/os is written in highly optomised assembly code and uses a lot of interrupts.

If I wrote code for the above functions would I have to be polling inputs? or is there a way to make much of this interrupt driven?

Is there too much overhead with using a high level language to code stuff like this? or should I be learning to do this on another platform?

any direction is appreciated!

I should have linked the above mentioned midibox stuff: www.ucapps.de

Checkout the DOUT, AIN and DIN modules