If you go the adaptor route, measure the voltage coming out of it; some aren't regulated at all, and will show more (sometimes MUCH more) than their output states. Ideally, you would also want to measure how much (if any) ripple is on the output; cheap ones can have a lot (you will need an o-scope for this measurement).
What a regulator does is keep the voltage at a certain level, for a given range of input voltage - so if the input voltage goes over or under the regulator voltage (within a certain range), the output voltage will stay the same (also, if the device being powered draws more current and such, it will keep the regulated voltage the same as well).
Servos are designed to operate from 4.8 volts to 6 volts max usually, so a 5V supply is at the low end of that scale (and you don't get the rated torque of the servo, either). I would actually look into an adjustable regulator instead of a fixed one, with proper feedback resistors to set it to six volts.
Also note that if you want to get the spec'ed current rating out of your regulator, you will need to attach a heatsink to it. If you need more current than it can handle, you can usually parallel regulators to give you more current (or, you can group your servos and power each set from its own regulator).
You can probably get by with a 1 amp regulator per 2-4 servos (depending on servo size, load, and distance from power supply).