Using an LED as a light sensor works but the method I used had two drawbacks:
It's slow because you use the LED as a capacitor and wait for it to discharge
it uses two data pins per LED.
If your LED's have tinted plastic lenses they will act as filters to limit the sensitivity to other colors. For LED's with clear lenses I have no idea if the color of LED has any significant effect on the color sensitivity of the LED.
When using an LED as a sensor, my experience has been that the LED will only respond to light from the same wavelength, and it actually has to be fairly close, so matched LEDs work best. A red LED can be triggered by another red LED of the same wavelength or reflections from its own light, but not from an LED of another wavelength. This causes problems if the LED acting as a sensor is in direct sunlight as it will produce sporadic results as the sun is full spectrum. Also as pointed out above steer clear from LEDs with a colored diffuser, you will want a clear LED.