I would agree with that.
You can use SD cards to read data from but they are too slow to write it in real time. External SRAM is the way to go. However, I am not sure if I2C SRAM will be fast enough. SPI will be faster, with parallel SRAM faster still.
I have bought some SAM chips to try this out but I have not got round to doing this yet.