Your idea for a "special" 328 as an ISP with additional functions is interesting.
I've been wondering about SD Cards recently. As far as I can see if you don't need them to be compatible with a PC you could write to them without using the FAT system using a much smaller Arduino code.
This, in turn, got me wondering about I2C EEPROMs which look like they would be much easier to use and much cheaper. Of course they don't have the Gigabytes of capacity that an SD Card has - but that may not be a problem for your situation. There seem to be plenty of DIP I2c EEPROMs which would be very easy to add to a project - and could be removed from the device at a later stage if required.
...R