Hi, thank you for trying to help. I am changing build.f_cpu in boards.txt file and F_CPU is taken from this property for IDE. What I didn't do I didn't change bootloader.OSCCFG value. Now it all works as expected.
To summarise (boards.txt setting):
// Nano Every in 16Mhz mode
nona4809.build.f_cpu=16000000L
nona4809.bootloader.OSCCFG=0x01
// Nano Every in 20Mhz mode
nona4809.build.f_cpu=20000000L
nona4809.bootloader.OSCCFG=0x02