I could add an offset to the error output to scale it up to a minimum, but not sure how I will deal with zero then. I am going to play around with gains some more tonight. I read some more today and I think that playing with the integral gains might help. However, the fact that it is not linear (meaning once it is moving, it can take a lower PWM than if it is still) is going to pose some issues, I think.
I think the original application got around this by using a slower PWM frequency. 1ms (12.5% duty cycle) is the minimum pulse width used. At 9V, that would mean the motor would see an average voltage of 1.125V. I doubt the motors really move at that low of a voltage... but it only uses that voltage as it approaches zero error, so it was moving to being with. From the signals I looked at, it seems to always start out at full speed and only adjusts as it is reaching the setpoint.
A 12.5% duty cycle with the analogWrite values is 31. The motor does not move at an analogWrite value of 31. I imagine that has to do with the responsiveness of the H-Bridge. Using the default PWM frequency (490hz on pins 3,9,10,11), the motor output is high only for 250us at 12.5%. At 50% duty cycle, I get 1ms which is the lowest the original PWM uses. I assume that this is why it begins to turn on around that point. The highest it would ever be is 2ms for 100% duty cycle. The original application goes up to 8ms.
You can change the PWM frequency, but it will affect the timers used by other functions. BTW, pins 5 and 6 are twice as fast at 980hz.
In the end, I think I am going to just have to come up with a software PWM method.