It's all to do with the way the ADC works. It uses a method known as "sample and hold". There is a small capacitor in the input, and the ADC "samples" the input voltage into that capacitor, then "holds" it there while it converts that stored voltage into a digital value. With nothing connected to an input pin, when the multiplexer switches the capacitor to sample that input, there will still be some of the old charge left in it - couple that with noise picked up by the pin, and you get a rough trace influenced by the previous channel that had any real input.
One good trick with this kind of ADC is to have one input dedicated as a purging input. Connect it to ground through a small resistor (say 100 ohms) and sample that channel in between each channel you are interested in. This ensures that the S&H capacitor is purged of residual charge between readings. The resistor is important to reduce current flow, but you want it small enough that the capacitor discharges in a reasonable time. Yes, you could short it direct to ground, but then you run the risk of too much current being drawn through the pin and killing the IO port (what happens if the pin gets switched to output and high? *pop*).
For instance, compare these two graphs. 4 channels, sampled in the order red, green, glue, magenta. The red channel has input, the others don't.
In this image all the other inputs are just floating and read in order.
However, in this image an extra channel is linked to ground, with a 10Ω resistor in this case (though that may be a bit small TBH) and is read in between each of the other channels. You can see how each channel has less and less of the charge from the red channel left in it.