Cleaner ADC measurements which of these ways is better

You might just try a capacitor first. The ATmega datasheet says:

The voltage reference may be externally
decoupled at the AREF pin by a capacitor for better noise performance.

But, it doesn't say what the impedance or suggest what value capacitor to use.

Plan 1 - Keep powering AVcc with same switching regulator that has 40mV ripple however use low pass (RC) filter and get smoother voltage curve into VREF. I'm thinking about R = 200R and C = 1uF, or play with values of RC chain so that voltage that goes into Vref does not drop too much,

That's worth a try. Like I said, I don't know the impedance but I assume it's much higher than 200 Ohms.

And actually, a fixed (and hopefully known) voltage drop would be OK. ...Your other solution was a voltage regulator to drop the voltage.

Or, you can use the internal 1.1V reference and a voltage divider on the ADC input (if necessary).