Are you missing something? YES. LEDs are current operated, not voltage. Each needs about 20 ma. What are you using for series current limiting resistors? 220 ohms are normally used. Each color actually needs a different current, but not really critical.
If you are trying to run more than 1 LED from an Arduino pin, no wonder you have problems.
Paul