Once you have decided to have an avatar (the default of course being not having one when you join) it seems you're stuck with having one. Yes you can change it, but there's no way that I can see to go back to not having one.
A work around, if I'm right, would be to save a blank bit of canvas in Paint but that seems an unnecessary pain in the arse.
Am I missing something?