I've seen the K20 ref manual and the SPI includes 4words deep FIFO for transmit and receive, moreover it supports 32bit SPI transfers. Try to use the FIFO - it helped us much with pic32 retrobsd Sdcard driver (I wrote a small routine for tx/rx with "enhanced buffering" as it is called at MCHP, DMA not used, but it has got similar performance finally).