And it doesn't sound like there is a problem with the amount of power
Well it sounds to me like it is a question of power. If you drop the resistor and it gets worse then it is likely that the extra current through the LED is causing the voltage to droop and that is affecting performance.
Infrared is kind of frustrating. It is hard to tell what is going on.
If you have a digital camera or a web cam then use it to look at the LED you can see it light up and judge the relative brightness.