You should be able to add 16gig of external memory
Huh? I don't recall CM3 being able to deal with more than 32bits of address space. Wouldn't that would require an MMU, which CM3 doesn't have?
Also, the datasheet says in it's "external memory interface" chapter: "up to 16MB per chip select" and "up to 8 chip selects."
It'd be close, though. The ChipKit MAX32 runs a version of BSD unix, and it's only got 128k of RAM. The SAM3X has "up to 100k"
(the retroBSD that runs on MAX32 may be pretty far away from what people expect when they ask about linux, though...)