Advice on a method to switch between USB and external battery power


I am building a project on a 5V arduino pro micro clone, similar to this sparkfun one

My project incorporates 2 switches. One is a power switch and the other is a locking switch. the idea being that when the locking switch is in the “unlocked” state the power switch turns the project on/off, but if the locking switch is in the “locked” state then the on/off switch is rendered useless. stopping anyone from turning the machine off when I don’t want them too.

I accomplished this using the method in the attached image. basically a power source is connected to the top end and the bottom end is connected to raw power. each switch has a pin connected to it so I can read which switches are active when there is power.

That all works fine.

The thing I’m trying to figure out is how to keep the ability to use this switch system if the project is powered through usb?

also will it hurt the micro pro if there is a external power source connected to raw and the usb is plugged in?

I hope that all makes sense.