I don't believe that there is a jumper on the Arduino.
You are wrong. On the earlier arduinos there was a jumper. But now there is a circuit that automatically switches to the external supply if one is present.
Couldn't you just connect your external +5v supply to the +5v pin of the Arduino?
I am afraid not, it doesn't work like that and it would likely cause a problem with the built in regulator, current equalising if the total from both is greater than twice the regulator capability, the same applies to dissimilar batteries.