Yes, it slows you down (but you're usually in the microseconds).
It's also why you need a current limiting resistor between the gate and the Arduino pin: it's as if you're charging a capacitor.
It is mostly an issue when doing PWM as the MOSFET spends some time partly on/off every cycle and that's when excess heat gets produced. The slower the on/off part the more time spent in that region, and the more heat is produced. So a high PWM frequency and slow on/off means the MOSFET is most of the time in the partial on region, and can get really hot really fast.
It is quite complex to calculate the time spent partly open (if you're interested, you'll have to do some searching for it yourself) - it's related to both the gate charge and the gate capacitance, plus of course the drive voltage, the gate resistance and the external resistor.
As a rule of thumb: the bigger current carrying capacity of the MOSFET and the smaller the on resistance the larger the silicon area of the gate, and the larger the gate capacitance.