Buck regulators are also "voltage regulators".
A buck regulator (switching/switchmode regulator) or switching/switchmode power supply is almost always "better" than a linear regulator or linear power supply. But it requires an inductor and it's harder to build if you're building the circuit yourself.
Most modern "larger" (high current) power supplies are switching supplies. I don't think there has ever been a PC with a linear supply.
A small low-power linear regulator is cheaper and easier to build than a switching regulator and and since the total power is low, efficiency and wasted power/heat is usually not a problem.
The 5V and 3.3V regulators on the Arduino are linear. (It's simpler and it saves cost). But it means you can't power a lot of "other stuff" through those regulators because they can overheat and shut down.
Linear regulators can also have less electrical noise which can be an advantage in analog applications such as audio amplifiers. But that doesn't mean you can't use a switching supply/regulator as long as it's a good design, and the switching noise is often above the audio range. A lot of modern audio power amplifiers do use switching supplies.
If you are a manufacturer the extra cost of the inductor and more complicated circuit is (at least partially) offset because the components don't have to dissipate as much wasted heat so they can be lower powered and it may not need a heatsink.
In the case of a full power supply, the main transformer runs at a high frequency in a switching supply so it's small, lighter, and cheaper than a 50/60Hz transformer. But those are nearly impossible to build yourself.