How to interface an SD card with the Arduino using less programming space

I am in the process of making an LED matrix game with the Arduino which takes up a lot of the Arduino's memory (22kb out of 32kb and rising.) I want to connect an SD card to the Arduino to save data from the game to it since it is portable and can be plugged into a computer. The problem is that the Arduino's SD library takes around 14kb of programming space which puts my program over the maximum amount of memory. Is there any way to connect an SD card to the Arduino and read and write data to and from it while using less programming space? 14kb does seem like a lot.

Note: I do not want to use the Arduino's eeprom or use external memory.

Consider an I2C or SPI EEPROM instead.

You may want to consider the SDFat library instead. I don't know how much space it takes, but I'm sure it's less than 14 kb.