Run function once for 500ms with single click, 1s with double click, etc

We need to see your whole program. For instance, all timing variables should be unsigned longs, but are they ?

Personally I would give your variables more meaningful names, for instance startTime instead of prevMill. You would also be wise to have a single currentTime variable copied from millis() at the start of loop(). This gives you a consistent point of reference for all time calculations in the program