Did you also have a pinMode(13,OUTPUT); ?
The led is driven by an op-amp and if the pin is floating the opamp can amplify the unreliable voltage enough to light the led.
If you don’t configure it as an output, you can use the pin as a hardware logic probe, jumpering pin 13 to circuit nodes of interest.