This capacitor is mandatory. This capacitor, the driver and the motor coils together work as a sort of buck converter. All Elements are needed to work properly.
The A4988 driver is shurly not your main problem - even if it's operated near its limits. Does the heat sink get hot?