sainSmart 12V 16Ch Realy Board, Arduino Mega, cheap MP3 player. Ground issue?

I have a sainSmart 12V 16 channel relay board with a 12V power supply going to the board. My arduino has a separate 9V wallwort power supply. I have the arduino 5V pin to pin 1, and Arduino GND going to Pin 20, as described by johnwasser. I have a cheap mp3 player that is powered by the 3.3V output of the arduino running through two diodes to get the voltage down to 1.5V. Things have been working well, until recently my power supply's status led began flickering and making a repeated clicking sound.

I disconnected the 12V output and still saw the dimming LED and clicking sound. I reconnected it and it seems to be working fine now, but later it has happened once again, and now its worknig fine again. So I dont trust it. I bought another power supply thinking that mine may be on the way out, and when I hooked it up, everything worked fine, except I had tremendous amount of noise on the mp3 player. So much that I couldnt hear the song playing. I reconnected my old power supply and all is working fine now, but Im weary of it.

Additionally, with my original power supply, if I connect my laptop by USB to the arduino, the noise comes through the MP3 Player.

Im assuming this is some sort of ground loop. I have begun researching because maybe I need to add an additional 5V power supply to the pin 1, to isolate it from the Arduino. Or perhaps I need to tie the grounds from some or all power supllies. can anyone help?

Audio is one huge matter with lots of studys about noise and isolating. There must be a kind of filter you can use, not sure if decoupling caps on audio Vcc is enough.

The flickering and the mp3 noise may be related or not. You should post a photo of the setup and the schematic. When ground planes are not robust then appear these problems. You must add decoupling caps everywhere there is a Vcc, and when connecting grounds use robust connections. Laptop USB also may be a noisy sources (it has it's own audio which may interfere with yours).