I agree, but if you wanted finer tuning you could disable that particular interrupt, eg.
EIMSK &= ~_BV (INT0);
That is also a single-bit operation.
And turn it on later with:
EIMSK |= _BV (INT0);
In this case I don't think having interrupts off for a moment will be a big deal.