Vishay TEPT5700 - Ambient Light Sensor problem

The typical scheme is 5V -> 10K resistor -> analog pin -> phototransistor -> GND (a "common emitter" scheme). This sets the pin high when it's dark and low when it's bright. I can't say I fully understand the math but it just seems to work better at typical light levels.

With your current wiring a larger resistor should give you more of a sensor swing.