Look at the blink without delay example, that is the basis of what you want to do.
You get the time now with millis and compair the time now whit the time you want certain actions to finish or change.
Also look up the concept of a state machine it is what you want to implement.