Depends on what you mean by "lose capacitance".
Instead of 10nF they will have closer to 0nF. It doesn't cause the dielectric to short, but instead become a high impedance.
I have been using some radio shack Y5U 10uf capacitors.
Y5U is the temperature coefficient of the capacitor. Y5
tells you the range for operation is from -30C to +85C. The U tells you that within that range, the capacitance can shift from +22% to -56%. (Hint: it is RARELY +.) This is in addition to the voltage coefficient, which isn't specified. (But for a thermocouple is irrelevant because the voltages are so low.)
Y5U is a class-3 ceramic, which typically means it is "peaked" to have maximum capacitance at room temperature, but then drops off very (very) fast. So even though you are at 0C, I would suspect you're losing at least 30% of the capacitance at the input. Which means, in addition to the tolerance band (10%? 20%?), you're probably around 5-6nF when you go to freezing temperatures.
You'd be much better off with a C0G 10nF cap (any rated voltage) since they are extremely stable with temperature.
I'm not entirely sure by what you mean "I just have some voltage dividers". The thermocouple amplifier is a pretty sensitive device, have you followed all of the design suggestions in the datasheet for your PCB?