You can make it do that. Best is a standalone design so no regulator, power LED, USB interface, just the chip, 8 MHz xtal & 22pF caps (or resonotor, or internal), 10K reset resistor, couple 0.1uF caps.
Go into power down sleep mode, any button press is diode ANDed to hardware interrupt.
On button press, wake up, read the button, turn on P-channel MOSFET to power MP3 module & speaker amp, select an input, when done playing reverse all the steps.
Some other MP3 modules:
I have several of these, have only tried one, was easy to use.
http://www.mdfly.com/index.php?main_page=advanced_search_result&search_in_description=1&zenid=26152301373bc97a159d507a7ac152bb&keyword=mp3
I just got a couple of these but haven't tried them yet.