Arduino Uno is turned on when applying 5V to Analog input

I would like to understand an unexpected behaviour of my Arduino Uno. With no power source connected, I plugged a 5V power supply to my Analog Input A0. Then, the Arduino Uno turned on, like if it’s connected to the power supply.
Is this behaviour right? If yes, why this happen?

That will happen, but it is wrong, not right, you should not do that.

Every pin has internal protection diodes (clamping diodes) from GND to the pin and from the pin to 5V.
So when you apply a voltage to A0, it will flow via such a diode to 5V and power the Arduino.
However that internal diode can only do 1mA, and you use maybe 50mA.

Your circuit or your design should never allow that a large current can flow into a pin. You can add a resistor between the power source and A0. That will protect the A0 pin. You can use 1k or 4k7 or so.