OK, given your schematic I see two problems to start with.
One is that the ground from the relay board is shown as connected to the ground from the logic circuits. While it should logically be the same ground as the input ground to the 5 V regulator, it needs to be connected to the 12 V ground at the input of the 5 V regulator. In fact, that ground and the 12 V to the "JD-VCC" must always travel as a pair from your 12 V source to the regulator and on to the relay boards. The ground on the relay board has nothing whatsoever to do with the logic boards.
Similarly, I am quite unable to locate an actual schematic of the version 3.0 NodeMCU. Diagrams of the 0.9, but not this one. I do not know what is the distinction between the "Vin" and the "VUSB" but I suspect that you should be connecting the regulated 5 V to one or the other, but not through the module. I note you describe connecting 5 V to both the USB socket itself and "Vin" on the upper module. Why?