You're doing some good work with the millis() based timing - but what are those delay() calls doing there? Get rid of them.
For your button code: do look into state checking (you want to run it only upon the button press, not again and again if the button remains pressed after that) and debouncing.