Port COM3 in Arduino IDE still available even tough I didn't connect it to the board

Where? Under tools -> ports? Or at the right bottom? Or both?

If you're using IDE 1.8.x, the right bottom shows the port that you selected last, it does not show an available port.

So, where do you see it? And which version of the IDE are you using.