Take a look here, lots of batteries & holders listed.http://www.dipmicro.com/store/index.php?act=viewCat&catId=549
5V plug in wall supplies also
What you power your project with depends on how long you want it to run between charges.
I made an RFremote control that runs a month+ on a 1000mAh LiPo with the charging circuit above to recharge it from 5V.
I added power downs sleep mode, created an interrupt from button press to wake it up & send the key pressed.
Its based on 8 MHz promini, so power LED is on all the time. Could build one up standalone to use a little less power.
The battery was from gravitech.us, had to ask for it.
(Pic is missing the MAX1811 on its adapter and the Rs/Cs that go with it. )