The arduino is a pretty small consumer of power. Say 100mA max, for typical usage. If you have a 12V source voltage, you're putting in 1.2W (12V@100mA) and taking out 0.5W (5V@100mA), for an awful 42% efficiency. But say you had a 90% efficient switcher there instead; it would be consuming .56W, so you would be saving a whopping .65W or so...
A GOOD buck switching regulator ought not have much in the way of "fluctuations" at the output, after all the filtering is done (including that done on the arduino.) An intermediate idea, if you have (say) 24V batteries, is to make tgw SW regulator for the arduino produce 9V at high efficiency, and the arduino linear regulator take that and smooth it further (less efficient, but less loss now that source and target voltage are closer.)