You can write and read back from an SD card without storing the entire contents in an intermediate array.
You can also append data to an existing SD card file with the appropriate SD.open() command so it is not necessary to read the whole SD card into memory to add to it.
You can't influence the storage format on the SD card so what do you mean by binary format here ?