Power Arduino from ATX power supply

If it does provide 12V, I don't see why you couldn't wire it as you said.

If it's >12V you can work around it by providing a "dummy load". Find some big fat power resistors and hook them up to 12V, ensuring you do not exceed their power ratings. I'd start with a small load, say 0.1A (120 ohm resistor, 2W or higher power rating) and see if that fixes it. If not, you may have a supply that looks for a minimum load on another one of the outputs (e.g., 5V or 3.3V) and regulates everything else in relation, so you will have to put a dummy load on 5V (maybe) to get 12V down to normal.

--
The Flexible MIDI Shield: MIDI IN/OUT, stacking headers, your choice of I/O pins