Further resolution...
I had included a 3-amp PTC resettable fuse in line with the 5V output going to my Arduinos... that fuse is getting REALLY HOT.
So what I'm pretty certain is happening is that the PTC is "tripping", resulting in a high resistance (PTCs don't actually open like traditional fuses) under the current load... which leads to the lowered output voltage. This would also explain why the PSU isn't shutting down... it is still seeing a "normal" load.
I've taken one intermediate step... my Arduino code now shuts off the control output to the servos if they haven't been commanded to change position for more than 3 seconds. This should be OK operationally speaking, and it brought the supply voltage back up to 4.5V. If needed I'll power them up once a minute or so and reset the positions to be sure.
Second step will be to take more accurate load current measurements and either replace the PTC with a higher current one or remove it entirely and rely on the PSU's internal overcurrent control. In fact, I should be able to estimate the current load by using the voltage at the load side of the PTC and its resistance spec when tripped.