If you want to go down from 12 or 9 Volts to 5 Volts and you do not need more than 200-300 mA, line regulators are just perfect.
If you are running on battery power things are a little bit different.
If your power source is AC, you should not worry about power losses in the regulator.
Going from 9V to 5V at 200 mA is just a power loss of 0.8 Watts. You will not even need a heat sink in that case.
At 100 mA it will be only 0.4 Watts.
The size of the capacitors depends on multiple factors.
At the input side I recommend at least 220 µF for every 100 mA. At the output side I always use 47 or 100µF if the output current is not higher than 200 mA.
Always use a 100nf ceramics capacitor in parallel on both sides to suppress high frequency spikes.
The size of the input capacitor depends on the voltage drop between input (behind the rectifier bridge) and the output.
The smaller the voltage drop, the larger the capacitor has to be. If you look at a rectified sine wave you will understand why. The capacitor has to compensate for the time when the sine wave is reaching the crossover point. And do not forget the voltage drop of the diodes in the rectifier bridge.
A DC/DC converter does not make sense in most cases when you only need a little bit of current.
And they give you high frequency spikes, which is not good when you have a sensitive circuit.
I once had to replace a DC/DC converter in a signal generator to get clean signals.
And you can hardly make a mistake when you oversize capacitors on the input side. They just cost more and need more space.
All my recommendations are referred to an AC power supply (transformer plus rectifier bridge)
Powering devices from batteries is a different world.