TX light won't go off until I touch the circuit

I've set up a simple circuit with one button with 5 v coming in and then a 4.7k resistor going to ground followed with (on the same pin on the switch) going to number 13,

Its only until I put my finger on part of the circuit the tx light goes off and then I'm able to use the button..

This is because you have not wired up the push button correctly, or the wires are not making proper contact.
See this for how it should be wired Inputs