Hello sdragou,
Verify that your PWMpin is not asigned to pins 5 or 6.
According to Arduino's analogWrite 'Notes and Known Issues', see analogWrite() - Arduino Reference
The PWM outputs generated on pins 5 and 6 will have higher-than-expected duty cycles. This is because of interactions with the millis() and delay() functions, which share the same internal timer used to generate those PWM outputs. This will be noticed mostly on low duty-cycle settings (e.g 0 - 10) and may result in a value of 0 not fully turning off the output on pins 5 and 6.
If that is your case, then, it explains the behavior of your project. Regards!
P.S. Remember also that the motor slows down for certain amount of seconds.