How do I accurately measure very close voltage levels?

[johnerrington]

.1% is achievable with the arduino ADC. - Which is 1 part in 1024.
You can get even better by oversampling as described here

skillbank: Experiments and projects with an Arduino microcontroller

Do you want to measure both levels, or just the difference between them? If the latter a difference amplifier like this would be appropriate.

Using Operational Amplifiers in your Arduino project

7: Important characteristics explained: Input Bias current, Input Offset current, Input Offset voltage, and "headroom" Fig 6 represents the input stage of a conventional bipolar op amp. You can see that the inputs Vin+, Vin- need to provide base current Ib1, Ib2 for the transistors. This is the input bias current. If the gains of the transistors are not identical they will not take the same base current. The difference between them is the input offset current. Suppose Vin+ = Vin- ; then Vout …