That's what I was afraid of, the resistor represents a load so it shunts the output to ground. You then measure the voltage with your meter with the + lead at the output of the regulator and the - lead to ground. Without any load, it's common for many regulators to have a rise in the output voltage.
The LM317 is an interesting device. It can be used as a constant current source instead of constant voltage. What it is doing is outputting whatever voltage is required (assuming it can of course) to drive enough current thru R1 and R2 to create a voltage of 1.25V at the adjustment terminal. So R1 can be chosen to create the 1.25V drop and then R2 would be replace by your circuit that you wanted to feed a constant current thru, such as a string of LEDs.
EDIT: For example, if you wanted to drive 20mA thru a stack of LEDs, you would use a 62.5 Ohm resistor for R1. This is a basic Ohm's Law calculation where E=1.25V, I=.02A and R is the unknown.