Because they share a common ground I imagine that's the issue,
No, in fact if they were not connected to a common ground that would be an issue.
It is possible that if you ran this without power to the USB, the the Arduino could power up by the parasitic powering of the direct connection to the digital pin. That is one of the reasons why a resistor in series with the data line is recommended. Also you should put the large capacitor across the LED supply as illustrated in reply #3.
Make those changes and I don't think you will have this trouble again.