A mosfet gate has a high resistance and relatively high capacitance. If you leave the gate unconnected, it will be somewhat random how much it turns on. However, if you connect the gate terminal to the source terminal then it should turn off. If you connect the gate terminal of a N-channel logic level mosfet to +5v or more relative to the source terminal, then it should be fully on. For a logic level P-channel mosfet, you need to put -5v on the gate relative to the source to turn it fully on.
From your diagram, I see that the connection from Arduino ground to mosfet source terminal (battery -ve) is missing. That's why your circuit doesn't work properly.