I am trying to get rid of delay() from within a 'for' loop, for all the good reasons discussed elsewhere.
What else is the Arduino supposed to be doing while this code is running? Where are you reading from the potentiometers?
If you are to get rid of the delays, you need to implement a state machine. Replacing delay() with your own do-nothing-for-a-specified-period-of-time() function is not useful.
So, at any given time, you should be in one state (3 LEDs lit, for example), and waiting for the appropriate time to transition to another state (4 LEDs lit, for example). The millis() function and stored times are used to determine if it is time to transition to another state.
The tricky part (not hard, just requires some thought) is knowing what state to transition to, and what action(s) to perform during the transition to that state or the transition away from the previous state.