If you increase the strength of the magnets, you will find that the no-load speed of the motor decreases for a given supply voltage, but the torque increases for a given current. You could increase the voltage to get the original speed back, but if you do that then you should current-limit the motor driver so that you do not feed it more current than it would have if it were running from 6V.