I notice that all reports are from Linux users. We sometimes see infrequent reports of strange errors like this that are later found to be specific to the 3rd party modified versions of Arduino IDE you get when you install it via a package manager (e.g., APT). If you are currently using an installation of Arduino IDE from a package manager, I recommend you install the official build of Arduino IDE and check to see whether the problem also occurs with that build. The download links are listed here:
There was a bug in Arduino IDE that produced a similar (but not identical) error:
However, that bug was fixed in Arduino IDE 1.8.6 so I think the problem you are experiencing has a different cause.
The need for Arduino to produce ARM Linux builds of Arduino IDE 2.x is tracked by the developers here:
A community member does produce unofficial ARM Linux builds. They are available here in case you want to give it a try:
It is easy to assume that our own personal experience is universal. I can assure you that is not the case with this particular problem. Many thousands of people compile their Arduino sketches every day for ESP32 boards using Arduino IDE 1.8.19. Yet, only a small number have reported experiencing this show-stopper fault. I just tried several compilations for the "TTGO LoRa32-OLED" board in 1.8.19 myself on a Linux machine (though not ARM Linux) and had no problems.
You are proposing an increase to the complexity of the Arduino IDE user interface solely for the purpose of a workaround for a rare bug. The correct solution would be to fix the bug.
Since there are no reports of the fault occurring with Arduino IDE 2.x, and even multiple reports of updating the IDE fixing the problem, I suspect that it was already fixed some time ago. If so, adding a "build all" option wouldn't even benefit you, since you would need to update to Arduino IDE 2.x to get access to the new feature anyway.