Hi, as I suspected, you are not using logic-level FETs. Most of the time you are getting away with it, but in certain situations, the FETs are not fully switching on and not giving enough output voltage to reliably light the blue and green LEDs. You are still getting away with it for the red LEDs because they have a lower forward voltage, i.e. they switch on at a slightly lower voltage than the green/blue.
The easiest fix may be to swap your FETs for some logic-level equivalents. They often have an "L" in their part codes, but to be sure, check the data sheet for the "Gate Threshold Voltage" figure. Logic-level FETs will have a maximum threshold of around 2V or less. Your IRF510 have a maximum of 4V, by comparison.
Another way would be to use some intermediary transistors between the Arduino outputs and the FET gates.
Paul