I'm not sure how much you want to spend.. but if it's a setup worth spending money on... you can get a switching regulator that can give you up to 95% efficiency with batteries.. and has even a lower-dropout voltage than LDO regulators.
They also have variable regulators with a trimmer pot built in, so you can just attach the input.. ground and output, then control the voltage with the pot on top.