@shawhuang22, Hi, welcome to the forum!
Please read How to get the best out of this forum, if you haven't done yet.
The answer to your question depends heavily on the library you intend to use!
Adafruit_GFX fonts do not support UTF-8 and have no degree symbol, afaik.
GxEPD2 can be used with U8g2_for_Adafruit_GFX which supports UTF-8 and has support for many more fonts.
See also example GxEPD2_U8G2_Fonts_Example.ino.
Jean-Marc