Should I control the high power LED or its driver?

Thanks for the replies!

Good point about destabilizing the circuit - makes sense now that you point it out.