Yes, that occurred to me too. And I think it should work. The right kind of encoder has both switches open at each detent, so no current would flow when it's idle. Then once the wakeup interrupt occurs, you could disable further interrupts and finish processing the encoder move by polling. Well, there are a dozen ways to handle that, but in any case the encoder would give you more precise control at whatever level of granularity you want.
And since you wouldn't need the watchdog timer running to do periodic wakeups, you could turn everything off and the sleep current would be almost unmeasurable. And of course it would only wake up when someone actually turns that knob.
But I'm not sure the ATTiny85 has enough pins to do all this along with SPI. The ATTiny84 mentioned earlier is I believe a 14-pin chip, available in all the standard packages, including DIP, and might be a better choice. I haven't studied the datasheet, but I assume its sleep current would be in the same ballpark as the 85. I just don't know what IDE support there is for the 84.