BenF, thanks for your thorough explanation.
It all started with this comment:
Grumpy_Mike:
Have you considered adding a software debounce ?
It is a lot more tricky on an interrupt pin, because the delay is in the ISR which is never a good thing.
I'd never call a delay() inside an ISR, but that comment got me thinking about what would happen if I did.