Need advice for supplying different voltages to a project

I made a PCB that drives 12V LEDs from an Atmega328p, so I supplied 12V to it and used a 5V regulator for the Atmega. Now I felt like adding USB support to it, and I hooked up a Silabs CP2102 USB to TTL converter board to it to see if it works how I intended. It did, however I noticed the regulator getting very hot while I had the USB connected. It doesn't do that normally, since it's pretty oversized (1000 mA) for the needs of an Atmega.

So I suspect that the 5V regulator tried to push power INTO the USB 5V rail, thus reaching its max current.
What's the best way to tackle this problem? I can't remove the regulator entirely, since the circuit should still work in the absence of a USB connection.

The Nano uses a schottky diode between USB supply and 5volt rail.
Other Arduinos have more complex circuits (p-channel fet, opamp).
Leo..