Wawa:
That buck converter could have a 2-4volt dropout voltage. Who knows.
That means you must power it with 8-10volt minimum to get your 6volt out.
Try three or four LiPo batteries in series.
10-12volt is better for the batteries as well, because battery current is lower when battery voltage increases.
Leo..
Thanks, I think this was it. I put another 18650 in series with the other two to push it above 12V and started getting a steady 6V out, with the servos running much better.
So much better in fact, that the full current melted one of my cheap battery holders, so I'll have to upgrade those now.