You can also obtain regulators with an Enable pin, and just turn it off directly.
Take any control pins to follow-on circuits low first so they are not phantom powered from high signals bleeding thru input protection diodes.
Example: LP3985
Selection depends on how much output current is needed.