A while ago, when I was playing with the Arduino UNO Wifi (and Minima), I experimented
with a setup like this. However instead of soldering the connection I soldered a switch
across it.
At the time I created a Variant of the WiFi, to setup to use the Native USB on the
RA4M1, which is similar at that point to the Minima.
There are a few threads on it, including:
I personally preferred running in that mode, as USB is faster. Does not require the board to copy the USB data over a non-fifo UART to/from the ESP32, which then did the USB transfer.
It also allowed the more complete Serial USB interface to work... Not sure if I still have that
variant sitting around here... Most likely way out of date
Good luck