There are situations where the running sketch can be so badly locked up that you can't just upload a sketch with the IDE. The programming port should be able to unlock it but sometimes it seems like Erase works better.
For incorporation into a permanent project where you don't want anyone to press the button, then either build it into a box or remove the button from the PCB.
Note that pressing Erase when there's no power doesn't erase anything. It's only a problem if someone can reach the button while it's powered up.