One more thought:
Turn off Wifi if you don't use it.
AR9331 (the heart of Yun) has a hardware bug where spurious RF ( WIFI reset ) emissions affect the USB PLL. Of course, Qualcomm Atheros will not acknowledge it as a critical bug as they found a software workaround for it.
http://forum.arduino.cc/index.php?topic=193540.msg1532112#msg1532112