Well, first... a speaker doesn't "require" wattage really. They all have different levels of efficiency -- that is, how much power it takes to produce a certain amount of sound -- but they will all gladly run on mW levels. Just... maybe not to your satisfaction in terms of volume.
OK, the hair-splitting is out of the way.
Here's a post full of recommendations on small chip-amps:
I have some little TDAs at home that I haven't played with, but I have built LM1875 and LM3886 designs. Both of those are waaay over your power requirements and supply rail budget though. 
If your supply is limited in terms of current (small transformer or battery-powered), you'll need to look into Class D. Unless you're very skilled, that means going with pre-built modules. Those are available all over the place. Check www.parts-express.com, and Google "class d amp kits".