I don't know the solution for your uninstallation issue but here's a workaround: On the Arduino download page you will see there are two different Windows links for each release, "Windows" and "Windows installer". If you use the "Windows" download then there is no need to uninstall or install the Arduino IDE, you just unzip the downloaded file and you're good to go. This is especially useful when you want to have multiple versions of the Arduino IDE installed on your computer because you don't need to uninstall one version to install the other.