Well you're thinking along the right track in some ways, but what about my example of the push button light switch, how do you operate a light (or any other device) if the system is broken?
Well, I thought if one arduino is down, I switch on the other one. But then if something else goes down... Well, I don't know 
Is there any other project similar to mine somewhere ?