Fortunately there isn't a lot of capacitance to bleed off. Figure worst case of 100pF for the diode and 10pF for the I/O pin, charged to 5V gives a charge storage of 0.55nC. Discharging through a 1uA reverse leakage current gives a total discharge time of 550us. That's not a very long time relative to pushbutton press/release times.