Yes, but your code already knows the input frequency, so at 2 Hz, only 1 sample is necessary. 100 Hz in --> 50 samples, 1000 Hz --> 500 samples, etc. So there would always be 2 Hz update in changing the output frequency, giving low jitter throughout the whole range 2 Hz -5 KHz input.
EDIT: In effect, you're keeping the response (latency) regulated to 500 ms in order to maintain low jitter. Note that at 2 Hz input and 500,000 us measurement time, 50 us of jitter is only 0.01% change in value. Originally, with 5Khz input you have 2000 us measurement time. So 50 us of jitter here represents 2.5% change in value.