hy does the motor work significantly better when used on the other pin
It's not about voltage, it's about current, and your output pins can't provide very much of it.
Even a tiny vibra motor from a mobile could draw more than twice the safe amount of current you could draw from an AVR's pin.
The 5V pin supplies current direct from the regulator which has a much higher current rating than the digital pins.