Happiness is SdFat with DMA SPI

OK - at least a 100MB test running OK is a good sign.

I reverted back to the original (ALAP and ROUND_ROBIN) configuration and tried to solve the issue purely through the matrix controller. These settings appear to be working:

(in spiBegin:)
  MATRIX->MATRIX_WPMR=0x4d415400;
  MATRIX->MATRIX_MCFG[1]=1;
  MATRIX->MATRIX_MCFG[2]=1;
  MATRIX->MATRIX_SCFG[0]=0x01000010;
  MATRIX->MATRIX_SCFG[1]=0x01000010;
  MATRIX->MATRIX_SCFG[7]=0x01000010;

(What this does is to stop masters from hogging the buses too much and keeps the bus slaves connected to their last master. There are even stronger options if this is not enough)

It may be an idea to apply both fixes (I tried, no errors after 100 loops) in the hope that if there's an obscure situation where one fix doesn't work, the other one does.