DrAzzy:
How about you build a bare-bones Arduino (or remove power LED, and probably regulator) and just use sleep mode so it doesn't burn power when not in use? You'd probably need to switch power to the servos with a FET though, IIRC servos have a non-negligible idle current.
It is true that I could have used the interruptions. 
In fact this project began as a simple quick project , and I didn't want to spend too much time on it, so I chose simplicity, used the Arduino Nano + an additional switch to shut it down.
Recently I changed my mind (actually I will give it as a present so it needs some improvements as compared to the "two switches variant") and decided that it would be better to avoid having this power switch, but I didn't want to do too many modifications to the code / hardware setup, that's why I chose to go for the OR gate + transistor. Plus as you say I would need a transistor to drive the servos anyway, so why not drive the Arduino through the transistor as well.
If I had planned to use the sleep mode from the beginning it would probably have been the best solution, but now that I have something working (except the power management currently under modification), I would prefer sticking to the original design for simplicity.
Thank you for this suggestion anyway, it would indeed make sense !