hearing unwanted beeps on my pc via arduino

I have an arduino duemelanove sending serial messages at a baud rate of 31250 (midi) over usb. The arduino is also lighting up an led using analogwrite every quarter note (so at 120bpm it is ever 240ms).

The weird thing is that every time the led gets illuminated, you can hear a beep on the computer (through the soundcard if you have headphones or speakers plugged in). This is totally bizarre to me and obviously there is some kind of interaction (via ground plane?) with the arduino and pc. At first I thought it was noise form the USB serial messages but it is actually sending bytes at a rate 24 times faster than the led blinks (for each quarter note of tempo there are 24 clock message bytes sent in between). This makes me think it is directly related to driving the led with analogwrite since that action directly syncs to the sound I am hearing.

This device is something I built for a music friend and he is on the road so I am trying to debug remotely. He is using it as a master clock for his laptop but has been unable to use it due to the beeping sound bleeding through.

Is it possible to disconnect the LED then test for the beep?

I am going to try to get them to do that. First though they need to test by powering the Arduino only by a wallwart and not via USB on a computer port. If the noise is coming through the ground plane then this will be pretty key for isolation. I am guessing I will also need some better decoupling on the circuit.

perhaps a large capacitor on the 5 volt supply might help suppress the noise. worst case you cant diagnose the problem, a quick fix might be one of these

http://www.circuitsathome.com/mcu/usb/usb-isolator