You could use DMA or not on any device or any transfer to a given device. SAM3X DMA is just an engine to move memory to/from a device register with the appropriate handshake/flow control.
I get twice the performance on SAM3X as on Teensy 3.0. I run SAM3X at 42 MHz and Teensy 3.0 at 24 Mhz. I use a lot of tricks on Teensy so DMA on Due is simpler.
I suspect there would be a slight improvement on Teensy 3.0 with DMA.
DMA could be a real advantage with a RTOS. If you run the SD task at low priority, more CPU would be available. Even better a task could wait on a DMA done interrupt.
Finally, I have the option in my SPI layer of using optimized SPI without DMA. It runs much faster than the standard SPI library but has none of the options.
I don't see what you have against SAM3X DMA.