Also I am using an N-Channel FET in the ground of the SD card
It may work fine, but usually such modules are always switched on/off by a high-side switch - ie a P-channel Fet in the VCC rail. Also mind the max drain current (up to 200mA for an SDcard). Switching complex modules low-side in the GND is quite rare.. (with the BS170 low-side you offset signal's log levels by +0.24V when Isd=200mA, probably not a big issue here but not a good design practice, though.. ;))