but can't for the life of me work out why its not doing what I want.
It would help with a better description of what you want.
There is a difference between a button being pressed and a button being pressed for a second time. You don't seem to be making that distinction on your code, do you need to?
What you want to think of is transitions, to detect a button pressed and button released. I think the just pressed is giving you the wrong thing to think about. It is that you need to detect and use that to trigger what you want to do.
That is clearer, thanks. What you need is a state machine diagram.
Draw what you want to implement and then just monitor the switches for the appropriate transitions. Having that will make the coding easy. As it is your code is a jumble of logic that won't implement what you want.