derniwi:
Is the only way to get this work using short wires?
Obviously short wires would be best.
If you can't do that, at least try to group the wires together closely, minimizing the area in the loop formed between any signal and the GND wires. The ideal physical wiring is a GND wire tightly twisted with each signal, with all GND wires connected to the GND pins on both the Due and SD card.
The other thing you can try is adding series termination resistors on the 4 signals. Values between 47 to 220 ohms are probably about right, though the specifics depend on the type and spacing of your wires. The wire used in CAT6 cable is designed for exactly 100 ohms.
These issues are common with the WS2812 (aka NeoPixel) LEDs, where a digital signal from a pin is run quite some distance from the pin to the LED strip. On my OctoWS2811 library page, scroll down to the "Signal Quality" section for more info, and some oscilloscope screenshots that show the issue with a real signal.