I've only just found out about these little DDS units, acquired one, came here, found the OP's code near the beginning of the thread and thought it would be ideal just to get the unit up and running for a first test. Despite doing all the obvious things (such as changing 'rotary' to 'Rotary') I had exactly the same problems down to the same error messages as the OP originally did when trying to import the 'Rotary' library, despite having been to Github on the link earlier in the thread to download it.
After running around in circles like the OP I finally realised what I had done wrong: On Github, I just right-clicked on each of the individual files listed and did a 'save as' to the Arduino/libraries/Rotary folder that I had already created for the purpose. A long time later it finally occurred to me to look at the files I had saved and they turned out to be web pages
So, back to the Github page, looked more carefully, found the download button for the single .zip file of the 'Rotary' library, downloaded it, extracted it into Arduino/libraries/Rotary (allowing it to overwrite all the earlier bad files), exited from the Arduino IDE and went back in, and was at last able to compile the OP's original code.
Incidentally, my thanks to the OP ('Nivar') for that code, which sped things up a lot for me - I didn't have an encoder so I modified it to use some extra switches: Increase frequency by step size / Decrease frequency by step size, Increment step size (as before) and I also added a Decrement step size button as well, for convenience.
I have to say these little DDS units are incredibly powerful for the price. Although originally for RF use, I took mine all the way down to the low audio frequency range and it could just as easily be used to make a precision audio sine wave generator.
Only one drawback with them, and that is that the current consumption is quite high - in fact, the crystal oscillator and chip both run quite warm, with the module taking around 100mA from a 5V supply, which means they wouldn't really be viable in handheld battery powered equipment.