What is pushing negative values? The sensor? The sensor produces a square wave, the frequency of which tells the colour. Or the colour intensity? This I know nothing of.
Just check what values you get when nothing is in front of the sensor. Define the condition according to those values. Fix your code so that when that condition occurs, zero is outputted to the leds.