can be a bit expensive
There's the rub. An LM2597 is more expensive than the ATmega328, and under "typical" usage, isn't that much more efficient than a linear regulator (switching regulators tend not to be that efficient at low loads, like the <100mA Arduino circuitry. )
You have to decide what you're really aiming for. A "low power" arduino aimed specifically at battery operation would be a good thing, but replacing the regulator is not all that you'd want to do, and the design would/should probably look different for a "longest possible operation of a low power circuit from 4AA batteries" than it would for a "I really need 1A@5V for the rest of my 12V car project.")