As Grumpy_Mike said, it cannot be fixed with the parts you have.
Lets look at what you are asking for a moment.
The motor has a current range of 0.3 A no load to 1.43 amps full load. So we will look more closely at a moderate load of 0.5A.
Now none of these calculations are linear so the simple example is only an estimate but it is close enough for you to see the concept.
Starting with 4.5 V power and a 0.5A motor with a moderate load.
Say you want to slow the motor down to 3.0V. The resistor needed would be:
- R = Volts/current
- R= (4.5 - 3.0)/0.5
- R= 3 ohms
The power dissipated in the resistor is:
- P= IIR
- P= 0.50.53
- P= 0.75 watts
You would need a resistor capable of 2 X the power so it doen't overheat. That would be 1.5 Watts
The little bitty end of your 50K ohm pot cannot do the job.
The part I left out for clarity is the current will change with load, so the voltage will change with load causing the motor speed will change with load (a lot).