The SERCOM works fine. Albeit with a 12MHz maximum SCK.
The SPI library is what slows things down. An individual SPI.transfer() ends up with a lot of overhead. i.e. gaps between bytes.
I don't see how you can say the SERCOM works fine and distinguish them from one another like that when the SPI library relies heavily on the SERCOM library and the reason the SPI library has a lot of overhead is because SERCOM doesn't have a block transfer function built in, and calling the write and read functions repeatedly in the SPI transfer function creates all that overhead.
I mean one could write the SPI library to do the SPI setup and transfer and such on it's own and never touch the SERCOM library, but it seems to be written such that everything else is done in SERCOM and trying to figure out how to move all the code into the SPI library without messing something else up (the SPI library now operates on transactions, so there's a lot of variables behind the scenes to keep track of that and the state of the SPI, and handle SPI during interrupts as well) was making my head spin so I just went along with it. I figured it had a better chance of becoming an official change this way as well.
Look at Rob's example. Mind you, this can be made faster by rearranging the loop.
Uh, Rob's example is my code with one minor change to how the loop is incremented. 
There should be overloaded methods for SPI.transfer() of a block. And if that is written efficiently, will make a huge difference to SD.
It won't, because the SD library as written does not call SPI.transfer() with the number of bytes to transfer, it just calls it in a loop. But I'll be fixing that today and will let you know how it goes.