Powering Arduino with regulated 5V source over 5V pin and using USB simultaneously

The only risk is driving power into the USB port. It's probably not a problem but if you want to be absolutely sure, take an old USB cable, slit it open, and cut the red wire. That will disconnect the 5V line from the USB port.