What is the power source? Is it the same as was used for testing?
What relay type and how is it being driven?
Possibilities include current draw when the relay operates causing a sufficient voltage drop for the board to reset or back EMF. A board schematic as well as details of all connections and power sources would indeed help answer that.
Looks like you need a reverse-bias diode across the relay coil
What’s the diode on the incoming 12V for?
Conventions are signals to GND are closer to the bottom and GND triangle points down. This will make your diagrams easier to read. The GND symbol pointing up is especially confusing.
Thanks for the input, I am definitely a electronics novice, the incoming diode is for reverse polarity protection. I will try the diode across the relay as suggested.
Depending on your PCB design, you may be able to solder this on underneath. Make careful note of the orientation. It is important. As @cedarlakeinstruments points out, the diode needs to be reverse biased, that is, cathode to VCC and anode to the switching transistor side as shown in the diagram in the first link.
A 1N4001 ought to suffice. Possibly you might get away with a 1N4148 if space is tight, but I will leave that for other forum experts to comment on....
One other observation, if I may, is that you are powering the Nano from 12V source, through a reverse polarity protection diode, but without any external regulator so approximately 11.4V will be present at the Nano VCC pin. While the Nano can tolerate that and it does have an on-board regulator, dropping from 12V to 5V dissipates quite a bit of heat. Since the regulator is mounted on the underside of the Nano, does not have a heat-sink and is located between the Nano board and your PCB, this makes it difficult for heat to escape which may lead to thermal issues. I can't tell from the photo whether you are using pins with spacing or just wire to attach the board to the PCB, or whether the case has any ventilation, but in an enclosed case, heat will just build up and may eventually result in premature failure. It might have been better to have added buck (DC-to-DC) converter to drop the voltage from 11.4 to say 6V first. The on-board regulator would then not have needed to work so hard.
@bigdealsalesfrank
The relay coil is a fairly powerful inductive load. In your circuit, it's essentially connected directly to the Arduino board's power contacts. When you open the transistor and turn on the relay, you'll experience a significant voltage drop on the positive power supply, which may restarts the Arduino.
One possible measure is to add a 200-500 µF capacitor directly to Arduino VIN and GND pins:
