Will this setup cause a fire?

The calcs above showed that with cat5 cable in order to stay within the cable's rated voltage and current you could carry the power but it would require using all the conductors. I wonder whether you could get away with using fewer conductors if you used solid core cat6 augmented cable? That has thicker conductors and would have a significantly higher DC current rating. Since you're carrying video and TTL data down the same cable I'm pretty sure that cross-talk would render AC completely out of the question so I assume you would transmit it using DC, and keep the DC voltage within the specs for the cable which probably means 50-60V maximum. Cross-talk issues may still kill you with UTP, though. You really don't want high currents running parallel to unshielded cables.

Again, this implies transforming and rectifying the power at the supplier, and doing DC-DC conversion down to 12V at the receiver side. It seems to me that the cost / complexity added by that would outweigh the convenience of being able to use standard network cable, but I don't know how interested you are in issues other than the cable itself.

You haven't mentioned where these cables would be used. Can we assume that they would be for permanent installation in a warm dry environment?

You might be able to get away with ethernet cable if you are willing to make enough compromises, but it doesn't strike me as the right cable for carrying this much power or that combination of signals. You really need a cable designed to carry the current and voltage you need, plus separate shielded cables for the video and TTL components.