If you have a high source impedance, and say you have (2) channel inputs on your ADC, you'll experience channel-to-channel crosstalk. The readings you'll now be getting from your ADC will be incorrect because of this crosstalk. As a side effect of the high impedance, you'll also lost some bandwidth to your input signal.
This simple isn't correct, crosstalk is not related with output impedance of the source, rather speed of switching internal mux circuitry.
Arduino input resistance for DC is about 100 M Ohm, it's not clear how fast OP needs to sample source, but even at hight rate S/H timing only 1 / 10 of ADC conversion timing, plus cap is only a few pF, so additional capacitor between input and ground may be used to solve a problem with lowering input impedance with fast sampling.
Still, I believe the main issue with photosensors is extremely wide dynamic range of ambient lighting, and that combine with relatively low resolution ADC (10 bits)http://en.wikipedia.org/wiki/Moonlight
The full moon is about 500,000 times fainter than the Sun.
there is almost 120 dB dynamic, which imply 19 / 20 -bit ADC