Yes, that is a perfectly acceptable way to have your program start assuming it has a sketch already loaded from a prior upload. While the reset is held to ground through a switch all the arduino I/O pins will be held in high impedance input mode, so just be sure your external circuits have any needed pull-down or pull-up resistor required to force their inputs to their 'off' or 'safe' states as your external circuits will have power but no valid control signals being sent from the arduino. You don't want random stuff happening to your external circuit while the arduino is in the reset condition.
Lefty