The newer u8g2 supports UTF-8. Once enabled with u8g2.enableUTF8Print() you should be able to print the degree symbol as expected with u8g2.print("°C"). Of course the font must contain the degree symbol (which should be true for u8g2 ending with "_tf")
Oliver