Glad it seems to have fixed the problem.
If I'm right, the voltage drops were not real. They look real because you are using, I assume, a multimeter.
If you looked at the voltages with an oscilloscope, I suspect you would not see the voltages you measured with the multimeter. You would see that the voltage was steady but switching on and off at high speed, and with different duty cycles for the different settings. This would look to a multimeter like different voltages.
Moving the } stopped the voltage switching on and off, making it steady with only that 180mV drop due to the LED current.