OK I see it wasn't optimal. Just to get you going, try this:
if(millis() >= off1 || ( digitalRead(buttonUp) == LOW && off1 - millis() < 4750 ) ) //see if it's time to turn off LED
If that doesn't work, revert to your original sketch and try this:
{
digitalWrite(led1Up, HIGH);
delay( 250 ) ; // NEW LINE - not nice but quick
off1 = millis() + 5000; //store var of now + 5 seconds
}
In both cases, the button should be pressed for less than 250 seconds or there will be a problem.
If all that doesn't help, then maybe a complete rething is necessary.