I agree that it may not matter if there are lots of interrupts per transition. I just wanted a way to reduce the interrupts in case it does matter in a particular situation. And I would say that polling hardly reduces overhead. If the encoder is idle, the polling still cranks along.
With a standard encoder you would have to accumulate 4 transitions in one direction or the other before reporting an actual detent change. Or for the other type of encoder it would be +2 or -2. So one switch oscillating back and forth shouldn't cause a problem in the first place.
But if you had an encoder without detents, and were counting every transition of either switch as a reportable tick, you'd have to do something to prevent flutter, and your method would do that. The only downside is you would have that dead spot when you change direction, but that's not a major problem.