If I were to take a SWAG I would guess the MOSI and MISO are reversed. Please post a schematic as to how you have wired it showing all connections, power and ground. Also include links to technical information on the hardware modules.
I know a possible way around the problem. I had the same issue using the HSPI bus, as pointed out the MISO pin can cause problems, specially during boot. I came up with a simple solution that didn't require any software changes and allowed me to use the default HSPI pins. I just connected a small PNP transistor in the MISO line. The base is connected via a resistor to the CS pin. When the communication starts the CS line is pulled low first, thus turning on the transistor. This way the MISO line is only active during communication and won't interfere during boot or with any other devices in the same bus.
When the master initiates the communication, the first signal sent to the slave is a LOW to the cs pin, connected to the GPIO 15 on the ESP32. This activates the PNP transistor enabling the communication through the MISO line connected to the GPIO 12 on the ESP32. When the master wants to end the communication it holds the CS pin HIGH. This makes the PNP transistor to cease conducting, cutting off the SD card reader MISO line from the ESP32 while its not being used. This in turn prevents the problems with the GPIO 12 remaining HIGH on boot and could possibly help solving conflicts with other devices that communicate on the same HSPI bus.
