If I go the op amp route, how do I determine what my cut-off frequency should be?
I would just try it as a simple non-inverting gain of one opamp buffer circuit first. If that doesn't give consistent numbers then you can always convert the circuit to a low pass filter circuit. Cut-off frequency can be as low as you want as temperature is a pretty slow changing process I would think low Hz values would be fine.
What we are trying to determine is if you value variations are predominantly caused by the higher output impedance of your divider or from just signal noise or a combination of both. So try the simple buffer first in my opinion.