As the switches are latching they will stay in the same state when pushed until pushed again.
Your current code only acts when an input becomes LOW, ie every other button push. Change it so that the current state is irrelevant and only the change of state, whatever it is, is acted upon.