Unfortunately although the SAM3 has an external memory interface the Due designers chose not to bring out one of the address lines and so IMO stuffed up royally._____Rob
Why? By the time you add external RAM to an Arduino, you're looking at a system cost comparable to one of the microprocessor-based boards (Beaglebone, Raspberry Pi, Xmega-A1 Xplained) that already includes RAM (typically, quite substantial amounts of RAM. The XPlained, with 8MByte is the smallest.)
I can't see me activating another SPI chips' SS pin using the port expander
I'm already using an SPI port expander, so I'll then be accessing two SPI devices at once?