What card holder? How is it wired? How are you powering it? Are you using the 3.3v power pin on the Arduino? If so, there may not be enough power available on that pin. Only 50ma available from that regulator, and some SD cards could use up to 100ma.
The Uno has 5v data pins, and the SD card has 3.3v data pins. Most SD cards have some type of logic level converter to do the voltage conversion, and a 3.3v regulator onboard for the power.