Start with the power supply noise. A linear regulator should be a VERY effective way of doing that - if you only have a 5V supply available, take a 3.3V regulator and run your amp at 3.3V. Amend R5 and R6 to still get your 2.5V bias.
Then you can worry about environment noise: that probably means shielded wires to your microphone, and maybe building the whole circuit inside a grounded metal case. That should take care of all environment noise.