I hadn't looked that close at the sparkfun schematic (too small). The VUB300 shows 6 or may 8 control lines - all would need to be muxed.
Some are output only - card detect, write protect - so just pass those along. The other 6 - 4 data lines, command, clock - would all need muxing.
https://www.sdcard.org/downloads/pls/simplified_specs/archive/partE1_100.pdf
Hmm, 5 of the 6 lines look to be 2-way signals. So maybe simple Mux is not the correct part, perhaps a low Rds analog switch that looks like a transparent connection would be better.
Ooh - two of these per master would be good.
http://www.intersil.com/content/dam/Intersil/documents/isl8/isl8499.pdf