I sympathize with this to some extent. I do see the warning causing a lot of consternation and acting as a "red herring", diverting people from discovering the true cause of their problems.
This warning was introduced along with the new system of scanning the libraries on-demand, rather than only at the IDE startup. Previously, you had to restart the IDE any time you made a change to the installed libraries outside the IDE. Each additional thing in the libraries folders adds to the scanning time, and the scanning is now happening much more often. So having non-libraries in the library folder has a harmful effect and there is no good reason for putting them there. So the warning encourages people to clean up their libraries folders.
Another reason to have this warning is that it's very easy to incorrectly installed a library when doing a manual installation. If there is no feedback then it might be very confusing to encounter the side effects of a library not being recognized without any clues as to why it wasn't recognized.
In the end, it is quite trivial to make the warning go away, so I'm not too fussed about it.
If you're still getting the warning, then it most likely means you didn't actually move it.