Will LDR sensor module latency be a problem?

LEDs work as a photodiode, but they are not optimized for speed. I have no idea if an LED would be fast enough for your project.

LEDs also respond only to light in a rather narrow range of wavelengths, typically 50 nm "bluer" than the emission wavelength.

Photodiodes are cheap -- buy them from any distributor (Mouser, Digikey, etc.). Or you can buy them as line and obstacle sensors from robotics suppliers like Pololu, Sparkfun, Adafruit, DFRobot, etc.