Motor and transistor

I connected the transistor directly to pin (the maximum base-emitter then current makes up 60mA)

Bad because the arduino pin should not be driven that hard 30mA at most.

with only 500mA collector-emitter current

How are you measuring this?
The only way for the transistor to get hot is if the current increases or the collector / emitter voltage increases. Try measuring the collector / emitter voltage under load / no load conditions then measure the current again under load / no load.
It could be that you power supply is flagging and causing an overall voltage drop thus reducing the base current.