Thanks Paul. I just got done reading the datasheet for the second time and I still don't get it. Everything there seems to be about driving digits, there wasn't anything about LED arrays. Unless I'm missing something? I'll be reading it again though.
I could also go with four multiplexers and a few logic chips using six pins on the Mega. I certainly have room for a larger board, that doesn't really matter. I just thought there would be an easier way, two or three pins notwithstanding.