I am not sure if this is germane but I am having a similar problem with an application that uses the AdaFruit TinyUSB library. However, I am not trying to use the debugger so this may not be relevant.
I can get my application to upload using the 2.4.0 version of TinyUSB but not with the current 3.1.0 version. The app compiles and tries to upload but then the serial port disconnects and does not reconnect again (as it does automatically with 2.4.0). Pressing the reset button does not help.