What are the critical considerations when selecting between buck regulators or voltage regulators?

No, so it will clearly be useless to discuss this further.

The Arduinos using linear regulators are really old designs. Recent ones use switching regulators as they became readily available.

As to the ESPs, they are 3.3 V systems and dropping 5 V to 3.3 V is generally not very efficient in a switching regulator (you need to use synchronous rectifiers instead of diodes) so the advantage over a simple linear regulator wasting the 1.7 V as heat is much less.