Smallest battery or set of batteries which supply 5v and 0.1A

You could use a single 1.2-1.5V cell of several types, and a very small boost converter to produce 5V.