I'm sure that what you are trying to do is possible. It should not matter whether you use a 8MHz crystal or 8MHz internal clock, the SD card does not have an internal time reference so it doesn't care what the exact clock speed is. I suggest:
-
With the SD card removed, run some sketches on your board to test it out. Include sketches that toggle the pins that you use to communicate with the SD card.
-
Check your connections between the mcu and the SD socket carefully, looking for open/short circuits.