zener diode to power an attiny?

RuggedCircuits:
If you've only got a 55mAh battery the shunt regulator you're building will be pretty wasteful. I'd go with a 3.3V regulator if you really want 3.3V (any reason you don't want 5V?) You can get a standard 3.3V regulator (LM1117 and others) or try to find one with lower operating current to get a bit more battery life.

--
The Flexible MIDI Shield: MIDI IN/OUT, stacking headers, your choice of I/O pins

I was mainly thinking zener due to space savings... although if I can get a 3.3V or 5V regulator in a small package (it only has to power the microcontroller so it wont draw much current) that would certainly do.

All regulators i've seen so far come in really large packages (which makes sense for medium to high current draw applications). If there's a smaller one you think would work well for this purpose I'm all ears.