I am not sure if I am driving it with PWM.
If you are controlling the speed with software, you're using PWM. And, it's (hopefully) PWM knocking 11V down to 4V "average". (We can't really count on a DMM to read average with PWM.)
It seems that one or two of them are turning with different RPM and as a result the vehicle seems to have a little drag to the right or left depending to the wheel with the lower RPM.
That's always a problem unless you have some kind of speed/direction feedback.
Stepper motors can be good or, you can mount the L/R wheels on the same axle, but I assume you need some way of steering. And without feedback there's always some slippage or something that keeps it from going perfectly straight. ...Your can can't go straight down a straight-road without feedback (you steering).