That is a bit of a mess as regards to capacitors.
Is the 100nF a ceramic type? It is essential that it is.
I would also recommend something larger in parallel with that capacitor like a 47uF.
Also you should have a capacitor on the input to the voltage regulator, again a 100nF ceramic and a 47uF in parallel.
The thing is that each individual manufacturer of voltage regulators has a minimum recommended capacitor. They might have the same part number but different characteristics.
On a product I designed the production line was stopped because some one in purchasing had changed the manufacturer of the regulator and it oscillated.
No not compared to 4V6. What do you mean by "better" anyway? I think you are misunderstanding this term.
How accurate is your voltmeter measuring this anyway? It might not overheat if you set the voltage at 4V6.