I made a device which works nice, powered from the outlet.
The problem is that sometimes the power has a glitch and the device does not get back to normal unless is plugged off and on from the outlet.
The idea is to work remotely, so I need to find a solution to stabilize it. Is there a way to do this?
You haven't posted any code, or explained what that code actually does, or how that differs from what you expect. So, all we can do is assume that your expectations are wrong.
A battery that gets charged from the mains power, and in turn powers the Arduino.
A standard power bank will do great for this purpose but make sure your project draws at least something like 50-80 mA or the power bank switches itself off (exact minimum current varies per power bank).
You also need to look at your circuit , if for example, those “glitches” cause an over voltage on an input , then you may damage the processor at some time .
Google “ brown out Arduino “
And “watchdog” for possible clean recovery techniques.