I have some examples in my webserver libraries (basically it's the same example but one based on ESPAsyncWebServer (the original from me-no-dev) and the other on the WebServer library included in the ESP32/ESP8266 Arduino core) and I have never encountered issues with reliability of SD or anything else.
I have to say I'm not using Arduino Nano ESP32, but I think is the same as "no-brand" ESP32 or not?
Note:
I would suggest staying on the core library because it's maintained, while ESPAsyncWebServer, although excellent, is no longer followed by the "real" developer (the one you installed is just a copy that follow the Arduino library registry requirements)