Larger resistors are more suseptable to noise. You could try reducing the impedance of the op amp's feedback network. Change the 100k and 1M into 1k and 10k, and change the 0.1uF to a 10uF. You'll probably have to use an electrolytic, so hook the - lead to the MIDI shield. It has the lower bias voltage. This will keep the gain and high-pass corner frequency the same, but should have less noise if it truly is picking up EM interference.
Also, so far you've done this with the MIDI shield hooked up. Have you tried it with your alternate source (I think your iPod, if I remember from the other thread).