The TCS34725 doesn't have it's own LED so I guess you're not using the sensor for itself but on a breakout board with other components. As you failed to post a link to that product we can only guess what exactly you're using.
As the LED is not integrated into the sensor the light from it has to go the subject and back to the sensor to be detected. If the subject is directly on the sensor this isn't possible anymore so the sensor must be kept in some distance to the subject to work correctly (at least for measuring the reflected light.
There may also be an error in the code you use to read the sensor. As you also failed to post that code we have no clue what you're doing there.