BTW: I have the same effect: Portenta H7 with ST-Link debugger connected: if I power cycle, e.g. I remove USB for ST-LINK, but still connected to Portenta H7 - I see the orange LED.
If Portenta H7 is powered, but ST-LINK connected - and not powered - you get orange LED.
It is fine to power also ST-LINK, or remove the debugger connection.
I guess: if ST-LINK is not powered - it holds the RESET signal low. So, your Portenta H7 does not come out of reset, the orange LED remains on - the PMIC is not configured (Portenta H7 hangs in reset) and therefore orange LED not going off.