I think you're right.
I tested the (19V 3A) setup with a multimeter and got 5.01V, which is encouraging. The 7805 has a small heatsink attached to it, and I can feel that it got pretty warm.
I'm loathe to try the 6V without the 7805 because I don't think it's regulated, and I can't afford to lose the servo (while there are several 7805s lying around).
Thanks,
-BBX