Debouncing capacitor

I agree with the first para by Paul_B. It is better to do this in software. Unless you have some tricky time-dependent code.
Basically an easy way do do this is to look for a change in the button status (with digitalRead, then wait with delay(50) and then check the button again. If the digitalRead is the same then the button is pressed and de-bounced. You might need to alter the delay.
Other will have their own way to do this. But a delay() is the simplest.

However, in your circuit, what you are doing is charging the capacitor via a resistor when you press the button. When you release the button, the capacitor will stay charged (at least for some time).
Try putting a high value resistor across the capacitor (say 10K) and that might help. That resistor will discharge the capacitor when you release the button.
This assumes that the negative of the capacitor is connected to GND (0V) which isn't clear in the diagram.