I am considering writing a lot of code to handle the different situations,
Don’t do that. Use the method everyone else does and have a memory buffer and just display that. Then all you need to do is to write the appropriate numbers into the memory to get any display.
See http://www.thebox.myzen.co.uk/Workshop/LED_Matrix.html