Powering ESP32 and externals

I have an ESP32 and a couple 5v servos, 5v fans and a 5v water pump.

I have the fan and water pump connected to a relay which is connected to 4 AA batteries (for a total of 6v).

Would it be okay if I connect the 5v servos to the set of 4 AA batteries as well? I can't imagine the extra 1v would be super bad. Or would it be better connecting the servos to the 5v of the ESP32?

Also, can I power the ESP32 with another set of 4 AA batteries? I was going to solder it to a usb cable and use that to connect to the ESP32. I was reading around that the ESP32 has a regulator of up to 12v. If that's the case, I should be able to use a 9v if I solder the power and ground to a usb cable, right?

To really help here, I think we need more information in the power consumption of the devices.
I think a problem you are going to have is battery life. AA batteries are fine but I would personally think that driving relays, servos, fan and water pump from that will quickly drain the batteries.
The ESP32 can also be quite power hungry in itself also.

On the specific points, many 5V servos will be fine with 6v power. But you ought to check the spec of the one you have to be sure. For the very short battery life you are going to see, it probably doesn't matter anyway......
The servo will need a PWM signal, so make sure you have your GNDs connected if you use multiple battery packs.
Is there a reason why you cannot power the EPS32 through it's USB connector? Or do I have the wrong ESP32 in mind.

In any event, with the detail provided, I would generally want a more "beefy" power supply design for this project.
Plug-in switched power units are very low cost and usually excellent - and arguably a lot cheaper than 1 set of 8 AA batteries.

If you post full details of the hardware, then we'll know what they all are. This means datasheets or product page links.