There's confused on what reliable means.
I'm not sure what you mean by "reliable".
Initially I meant reliable in regards to the regulator's output and the effect on the other components on the Arduino board, namely, the ATmega328. You are operating the regulator outside of its specs. Its output will not be reliable. If board's temperature changes its output will change. If the load on the regulator increases, its output will change (go down). If the input voltage drops so will the regulator. The whole reason for using a regulator is to prevent this change from happening.
Because that's what my question is about: since everything is working well right now, will it be reliable in the long run?
No. At some point you will see problems. Just because something works today doesn't mean it isn't damaged and will fail later.
but you also say that it "probably" won't damage the microcontroller
What I did not bring up before, but you should consider are other aspects of the ATmega328.
For example, the Absolute Input Voltage for any input before damage occurs is rated as 0.5volts + Vcc. In your case, you put 5volts into a regulator and you are getting less out; probably 4.4volts. So 0.5volts + 4.4volts = 4.9volts. If you are applying 5volts on Vin and that same 5volts to any other pin on the ATmega328, you are going to eventually damage it.
This is where "probably" comes in. You're doing a bunch of borderline things and asking for a definitive answer. So I will just keep repeating myself:
Do not put 5volts on Vin.