LED Matrix glitchy pixels and it stays on after unplugging

Correct

If any chip which is not powered is receiving a signal on one of its other pins from an external circuit that is powered, phantom powering can occur. It's bad for 2 reasons:

  1. The ESP's data pins can only supply a few milliamps. If more than that flows, they can be damaged. Powering all those LEDs, even if it is only the red LEDs and they are only shining dimly, takes more power than is safe for the ESP's data pins to source.
  2. Phantom powering is possible because all chips have ESD protection diodes built-in. Without them, they would probably be destroyed by static before they ever got used. But the side-effect of having those diodes is that if a chip isn't connected to a power supply, but one of its other pins is connected to a circuit that is powered, current will flow from that pin to the chip's power pin, powering it. The ESD diodes are not designed for that purpose, and this can damage the whole chip.

No, it doesn't happen because when the PSU is on, the voltage at the chip's power pin is the same or higher than the signal from the external circuit, so no current flows through the ESD diodes.