I recently started programming ESP32 using Arduino tools and Visual Micro with Visual Studio Community 2019. My build times for a relatively simple sketch were around 8 minutes. Totally unacceptable, takes half a day to tweak anything. I have traced part of the problem to real-time protection from Avira, you think it would learn after a while that VS is not a threat but, no I had to manually exclude everything ESP32 related.
So it went from 8 minutes just to 2 minutes. Then I noticed I had option set in Visual Micro for library deep search, after disabling it, the compile failed on not being able to find libraries, and a lot of them, as ESP32 uses A LOT of code. So I added missing libraries manually with AddLibrary option for Current Platform, ended up with a long list of includes (23), mostly WiFi stuff. And...
What do you know it takes only 20 seconds now!!! I'll tell you what, yes, it is annoying to add all required libraries manually, but I will do this any time to get 20 sec compile times.