External interrupt fires early?

fungus:
Did you try setting the pin as an input and using the pullup resistor?

Yes I did, it didn't make a difference.