I had this issue and wrote a 7-Segment display routine for the graphics LCD, basically an array of chars that hold the 7-Segment info bitwise. Some bitwise manipulation gave the segments that needed flipping, instead of refreshing the entire screen I just changed a few segments. I was using the nuelectronics.com touch shield cheap (£35 for a 2.8" touch lcd) but the refresh is insanely slow.