DS18B20 self heating or what?

Your experiment does seem to show a small effect of fan cooling. What is difficult is to assign a level of confidence to "self heating" (which could certainly explain the observation) or to decide whether the effect is worth worrying about.

If self heating is important, then it was probably taken into account when the device was designed and calibrated by the manufacturer. Can one do anything about self heating? Is the temperature measurement more accurate if you use the parasitic power scheme versus constantly powered? At this point you don't know because you need another, independent method to measure the ambient temperature accurately.

One possible conclusion is that if you want more accurate temperature measurements, you probably need a different type of thermometer.