It sounds as if, as well as being at the limit of how accurately it's feasible to measure an analog signal, you're also at the limit of how frequently you can take those measurements. This is not encouraging, and you may be constraining yourself into an intractable problem. I still feel that using a single brightness reading requires that reading to have unrealistically fine resolution and interpolating from multiple readings would enable you to get a better resolution.
One way to relax the precision requirements would be to reduce the size of the scanned area. That would mean you have even stricter requirements to get the scanner positioned in the right place and I don't know whether that's feasible.
Is it possible to give a more general explanation of what you're trying to achieve and what overall system this solution has to fit within? It may be possible to apply some lateral thinking, such as applying a vernier technique using multiple sensors, but without any idea what the thing is you're trying to measure the position of, it's hard to know where to start.