I had set port speed to 115200 and accidentally sent one byte per loop over the line. I then opened the serial monitor which nearly froze the IDE. With much patience I could set the speed to 9600 which made it responding again.
Pulling the USB caused a crash last time I tried so I didn't this time. 
Yea, I believe this a problem with either the library we're using for serial communication or the FTDI drivers themselves. Which operating system are you on?
PPC OSX 10.4.9
I don't know Java but my guess is that it gets an event everytime there is something on the serial port. Since its only one byte but coming at max speed its the overhead that kills the app. Maybe a 'dead time' between reacting to such an event might help.