Don't forget it's not just about the Arduino pin. The LED, without having a resistor to help provide a controlled current source, can draw a larger value of current than it itself can tolerate... and then you get HEAT and ultimately release MAGIC smoke out of your LED.
You can drive an led like your grandpa drives his Buick on sunday or you can drive it like James Dean drove his Porshe 550.
You pick... but one of those ways has your LED living fast and dying young.