I'm actually ok with the Arduino part of it. My plan was to use the Attiny85 and put it all in a little inside pocket, but the Lilypad could be a better fit.
Connecting everything, adding switches etc. should be fine too. I plan to use a Multiplexer/Demultiplexer to control it.
The part I'm struggling with is the design of the "buttons" themselves. I don't really have a good idea on how to make those.
Looking at a shirt like this, for example -
http://www.nitelifetshirts.com/nl424nltransformers-decepticons-p-48.html, it appears that there are 4 sections that light up, and each one is wide and flat. It also lights up homogeneously (i.e. it's not one bright spot that fades towards the edges). No idea what kind of material is used there, but if it's something that can be purchased and customized, it seems pretty ideal.
One big difference, though, is that I need mine to also be press-able, either simply by touch or by physically slightly pushing them in like buttons (touch would be better).
Hope that makes my question clearer, and really appreciate the help!