I need to measure very small distances (cracks in a wall) and I was wondering how others here would go about this? Is there anything small enough I can slide into the crack and read with an arduino (such as a tiny IR sensor). If not could I use a stepper motor somehow to measure how many steps before a lever encounters resistance?
We used to use gnats hairs for measuring small distances
Are you trying to measure width or depth?
Both I suppose, but lets start with width.
Some digital micrometers have serial outputs.
You could nail/glue the jaws either side of the crack, and read off the serial port.
That's a fair suggestion, but I am looking for something a bit less invasive (no nails or glueing), ideally I'd like to hold the arduino (and whatever I need to measure distance) but to the crack and measure it without glueing or nailing.
What sort of distance are you trying to measure, and to what resolution?
Distances are typical in millimetres below 5, and I would like the resolution at 100 micrometres
Those requirements seem pretty tough to meet, and may not be feasible at all.
Two approaches you might try are:
1/ using a mechanical probe which can drop into the crack - depending on the shape of the crack and the probe you could arrange for it to measure either the width at the surface, or the depth.
2/ Using an optical sensor and flow processing to pick up parallax changes as an optical pickup pointing almost parallel with the surface is moved across the crack.
Do you know of any existing sensors that solve this sort of problem? If so, I'd suggest you try to use the same approach that they use - whatever that may be.
Pipe dreams can be so relaxing. 
magnifying glass with a scale ?
Is there any point measuring such a "random" thing as the width of a crack in a wall to .1mm? (Have I got that sum right?- 100 micrometres is 100 x 10-6 which is 10-4 or 1/10 of a mm>)
Move the probe a gnats whisker higher on the wall and the shape of the crack will certainly make for a new reading. I doubt if you'll have any kind of repeatability, and I'm assuming you would want to monitor the width at a certain spot. Read it again a while later, holding the probe ever so slightly differently and I reckon your reading will be different by that tolerance, and you might conclude the wall has shifted when it hasn't.
Indeed I'd wager that the very act of inserting the probe in the crack, no matter how gentle you are, will dislodge a crumb of wall wider than the accuracy you're trying to measure.
This is a serious challenge you've offered.
I doubt you can do it with an arduino since the system will require a fair degree of "interactive intelligence" to interpret where the start and end of the crack are to be determined. In all probability there will be some degree of degradation of the edges of the crack that will require interpretation.
I'd suggest you consider an optical microscope that can be placed over the crack, this feeds onto a PC or laptop screen, with superimposed dimension scale. The scale is determined by the x-factor of the microscope. You then visually determine where the start and ends of the crack are and read off the scale dimension.
Calibration of the system is easily set by using spaced lines and setting the dimension scale accordingly.
I see Jimbo also has experience of measuring with parts of a gnat XD