I know in the one sketch that I used to read files from USB drive, I had code in it like:
if (!msd.connected()) {
if (msd.connect()) Serial.println("MSD Connect");
}
if (msd.connected()) {
That is, you don't try to do a connect if the drive is already connected.
You might try something like that to see if that helps or not.