often what you are picking up is stray power line noise (50 or 60Hz depending on your country as well as multiples there of due to distortion products)
Good to know, very interesting!
I meant ground when I said 'other probe'.
Oscilloscope probes are in a way "differential". This means they measure the difference between the signal on the probe and ground. Note I put differential in quotes because there are true differential probes which measure the difference between a positive and negative lead (negative is no longer grounded). So when you aren't touching the probe, the probe by itself is picking up the mains noise. When you grab the ground connection, you provide a better antenna for the ground connection. This shunts more noise into ground and serves to better offset the noise picked up by the probe itself.
As an aside, it is important to remember that the ground lead is ACTUALLY connected to ground. It is VERY easy to kill a scope with ground loops, which is why for power supply work a true differential probe is often recommended.