Set the environment variable DevMgr_Show_NonPresent_Devices to 1 so that you can see disconnected devices when displaying hidden devices in Device Manager.
By that way you can “remove” disconnected devices. It is a good thing to do a clean up in registry too.
Disconnect devices. Remove every COM port disconnected. Connect again.