After thinking on this, the direction I am going to try is to use avrdude from an R-Pi Zero to do both serial booload and ICSP from a relatively small handheld but power hungry setup. It will have spring probes on both ends. I modified a little level shift board I was using with my Uno that does the ISP sketch.
The exact details for running the uploader are yet to be worked out, but for me, it will probably involve a remote login and then running a Python program with a loop that waits for a switch and then operates a makefile rule based on the working project directory then does that all again. It can build from source (I use GitHub), so on the ICSP tool's host (samba share or whatever) I can even edit it and don't need to keep track of what SD card goes with what project.
Power is a problem, I will need to carry a car battery around to run such a thing in the back yard.