That's a very interesting and sensible idea.
Would it be sufficient to use the PC to upload the source code to the SD card and then just store the card on the device. And put it back in the PC if you every need to downlaod the code in the future. Indeed it might be possible to use the SD Card as the PC's source for the program code from the very start of the project. That way you also keep all the variations of the code on the SD Card.
I have heard that the SD Card library uses a huge amount of the Arduino's SRAM so it might not be practicable to use the Arduino to write/read the SD Card.
...R