it seems suspicious it stopped working while attempting to use another serial based peripheral.
Did you try connecting w/o the display?
ie upload sketch w/o display - unplug from computer - plug in display - use device - unplug display to upload sketch later.
Multi-use pins seem to be somewhat commonplace. The esplora may not make a good game controller with display for a computer. But more aimed at being its own game platform, that happens to be able to act as a controller while plugged into a PC (? - unless a display is connected).. I don't have that display - I also got rooked at radio shack on the seeedstudio display that wont work with esplora w/o some significant hardware addition/changes to it.
Also - IIRC you had to do something different if you run the mouse demo and want to upload a new sketch - because of a CDC/HID reason I don't know if I comprehended when I seen it.