You're seeing the voltage from the charged caps, try a bleeder resistor (47k ?) from the top of the caps to GND.