What do I achieve by using delay() in terms of power consumption, efficiency etc.?
Nothing, nothing, and nothing.
Should I stick delay(100) just for that sake, or does it hurt my program somehow?
No, it is not needed. It won't hurt anything, except that it will make your switch less responsive.
Is it better to delay main loop as much as possible, or as less as possible?
As little as possible.