Interference problem with slotted IR sensors

Something is wrong with your setup that I can't see. You will need a scope to troubleshoot it.

However, the 3.6A current limit is inherent to the design of the DRV8871 and can't be increased. It will go into overcurrent and/or thermal shutdown with that motor. Data sheet here: http://www.ti.com/lit/ds/symlink/drv8871.pdf

This driver can handle 5A peaks. I have been using this one for 7A motors for about two years, and have never had any problems.