The arduino, like any digital device, is only capable of reading a logical '1' or a logical '0' from a digital input. It has no concept of anything else in that context, so it must choose one or the other. You may think you have nothing connected to the input, but in reallity lots of things are capacitively or inductively or resistively coupled through e.g. the circuit board substrate or the very air surrounding the chip. Those things are usually negligible and drowned out by the valid signal when you've got something deliberately connected, but if not then those other things can cause the input to drift towards one or zero depending on what's going on around it. The same goes for the analog inputs. If you touch your finger to one of the analog inputs with nothing else attached you will cause the readings to change, for example, since your body has measurable resistive and capacitive proeprties.
Think of it like a weather vane; if you have a strong steady wind you will get meaningful readings. If it's mostly calm but with gentle intermittent breeze, though, it will just drift around rather randomly. Connecting a pullup or pulldown resistor or a valid TTL signal will provide that strong steady wind.