I had this yesterday exactly as you have described, that is accepted the invitation to upgrade the IDE to 2.3.3 while a download of the "package_esp32_index.json" was active.
I did lots of fiddling so I am not exactly sure what worked, but the final (and drastic) stage was to go into the boards manager (maybe in a second instance of the IDE) to delete the ESP32 core (in my case it was 3.0.4). This appeared to put the deletion into a queue behind the activity "Downloading index: package_esp32_index.json". I then, a bit later, opened a new IDE window from the old and there was no sign of the download attempt. I then, of course, had to reinstall the ESP32 core (about 10 minutes).
I hope there is a better way. Closing and re-opening the IDE by itself achieves nothing because the (stuck) state of the download appears to be cached.
I also hope that some effort is put into speeding up the package_esp32_index.json "download" and it is difficult to understand why it takes so long.