Yes - that will save power - but the savings are nothing compared to the power wasted by the external components of the Arduino (such as the regulator, USB/TTL converter and power LED).
If you want to go low power, dump the Arduino and get a barebones ATmega (or ATtiny for even lower power consumption). That alone will drop you from 50-80 mA to about 5 mA. After that lowering clock speeds and so can shave off a few more mA. Sleep mode can get you to the uA range (while an Arduino board in sleep mode still uses some 50 mA).