I googled for ATX power supply specs. They do output 5V regulated so that should be able to be used to power the arduino directly with no voltage regulator in between. Also, there are +12 and -12V leads. Taken together that's a 24V potential.
Yes, you do have to short pins to get the power supply to turn on. Here is a site which discusses this:
http://www.duxcw.com/faq/ps/ps4.htm