First of all a IRF510 will not operate properly with an arduino output pin because it is NOT a logic level mosfet. A logic level mosfet gate can be fully turned on with +5vdc, a non logic level mosfet requires => +10vdc to full turn on.
As far as your failure to turn off, I would want to see a wiring drawing of how you actually have things wired up before guessing. First impression is that 100K ohms is awful large to try and discharge the gate capacitance.
If I'm understanding correctly, what you're trying to do is power off the Arduino, sending a control signal from the Arduino itself.
This will not work with an N-channel fet. The Arduino ground is connected to the mosfet drain. The Arduino output pin, when set to low, only goes to its own ground. So the mosfet gate is still at the same potential as the drain.
A P-channel mosfet in the positive supply line might work.
A better approach would be to investigate the sleep library.