@war_spigot:
With regard to tactical vs. structural optimizations here I have one example where I blundered:
Have a look at my removing flicker experiments Removing Flicker | Blinkenlight . I used the macro processor for loop unrolling. A somewhat agressive optimization technique. Gives ~20kHz PWM frequency with 32 different levels. Of course it looks as if there is not more room for optimiziation left. It consumes significant memory and utilizes the cpu at 100%.
However there ARE better algorithms. I did not even think that there might be better approaches but there are. Right now I have a PWM on the Arduino that outputs 256 levels at all 20 pins with ~20kHz while keeping cpu load below 70% and consuming less memory. However the optimization technique to pull this one off is at least as aggressive as the macro approach 
Figure this one out if you want to learn about optimization approaches.