You've got a delay(20) in there. Your minimum response is over 20 ms....
It'd be smoother to loop while ( now - startPress > 10 ) true after 10 ms continuous contact, Arduino can check that button 8000 times in 10 ms so why have it do nothing for 20 ms?