I find 0.8mm to be the most generally useful size, and a set of 0.6, 0.8, and 1.0mm covers almost everything.
(you need the larger size for those square-pin headers, and the smaller size is nice for vias.)
0.8mm is also almost exactly 1/32 inch, and/or a #68 drill.