Assuming the source of the FET is connected to ground you need a pull up resistor of 10K or so on those gates.
Yes a p channel FET would prevent the inversion but you still need the pull up.
You also need current limiting resistors. What current are the LEDs drawing?