I dont have access to the dataheet right now but from memory a 4N35 will deliver about as much output current as you put in. 12v @ 500mA is 6W not 11.5W. Darlingtons such as tip121 have high saturation voltage so they run hot when passing significant amounts of current. You can try reducing the 470 to 220 ohms but I doubt it will help much. Either use a heatsink or use a mosfet with gate pulldown resistor instead of the tip121.
I assume you are driving the 4n35 output and that resistor from the 12v supply. 220 ohms is ok but could dissipate as much as 0.5W if the 4n35 were to turn on fully. I suggest 1k. See my previous response for further advice.
Ps im assuming the resistor is in series with the 4n35 output eg between the 4n35 and the base of the transistor. That's where it should be.
How do you go about connecting a flyback diode? Anode of the diode to the negative terminal and cathode to positive terminal of the motor?
And what kind of diode should I use?