SOLVED: Problem when plugging heavy load devices on the same outlet as Arduino

There are several reasons an Arduino might reset, one of them is power drop-out, but its not the only one.

The reset line might pick up EMI and trigger from that - adding a 1k pull-up on the reset line can really help prevent that.

Large currents switching nearby could induce voltage spikes directly into the Arduino circuit via electro-magnetic induction - without knowing the Arduino circuit's layout and what is connected to it its hard to judge this.

Poor PSU isolation could mean the Arduino is riding at a significant fraction of mains voltage (but at high impedance), so that when you bring an earthed object (yourself) nearby there is significant capacitive coupling - if this is picked up by the reset line, could reset.

We know there are heavy currents switching and presumably some voltage spiking (from the switch arcing). I'd like to know a bit more about the layout of the actual circuitry on the Arduino.

You mentioned:

Some large radio antenna on the nRF24

How large? That could be what's picking up voltage spikes.