But in order to take readings, you still only need to set the output once. So that is not a valid reason. How did you make the resistor selection (since you identified a possible problem there)? What is the voltage at the analog input? How much do you expect it to vary? Because, it has been suggested that the kind of circuit you have there may not work very well:
https://forum.arduino.cc/t/k-type-thermocouple/118746/13
In some circumstances, the first analog reading is inaccurate. Some people take two readings and only use the second one for this reason - technically it should only happen when changing analog ports, but it may happen on the very first. It's kind of complicated, but you could try taking one analog reading before you sample the next 20, and just throw the results away.
When you are troubleshooting, it would be better to print the raw reading, than the processed numbers.