ESP8266 not booting when connected to arduino nano with tx-pin

---I'm not quite sure I posted in the right subforum, please let me know if this is wrong.---

Hi, a bit long but I think interesting thing about the ESP8266/arduino nano
My esp8266 (nodeMCU from aliexpress) is connected to an arduino nano through a serial interface (esp TX ==> ard RX and ard TX ==> esp RX). This is done via a custom pcb. Both communication lines are also level shifted through a bss138 N chanel mosfet. I also added a mechanical (sliding) on/off switch on both lines.
The ESP is always on and the arduino is toggled on by the esp when needed. So when the ESP is booting, the arduino is not powered on.
The serial interface is needed for the two devices to talk to each other. The ESP TX to arduino RX is the most important however.
When the switch on that line is "on" (so when the esp TX is connected to the arduino RX), the ESP will not boot. Apparently, the esp TX pin is the GPIO1 pin, which will go high on boot and will fail a boot if pulled low during boot.
The low side of the level shifter is pulled up to 3v3 and the high side of the level shifter is pulled up to 5v all the time. So I thought this would take care of that problem.
But apparently the arduino RX pin is connected to ground when not powered on, because i can't get the ESP to boot.
When I troubleshooted the problem, I found that I could boot the esp8266 with the switch on the TX line turned "off", and switch it back to "on" when booted. From that point forward the esp will boot on a "esp.restart()" (arduino IDE) code without a problem. Removing the power and powering the board again, doesn't boot the ESP.
Now my problem: The ESP should be always on, so if I just manually turn the switch "off" on boot and toggle it back "on" after that, it should work. But that would only work if the ESP truly is "always on". Any power outage over half a second (i got some capacitors in there) might require me to go and manually reset it. The PCB will be mounted and closed of in a watertight electric box very much out of reach, so that's not really an option. A big problem is that I never tested the communication between the two and now have a working PCB, so I can't really swap parts in and out very easily (but if needed I can try).

Does anyone have any suggestions/remarks?

This is the schematic (Don't mind the labels, they are misleading. The 5V side is indeed the nano side, the 3V3 is the ESP side)

This makes no sense:
"But apparently the arduino RX pin is connected to ground when not powered on,"

The Arduino RX pin should be floating, I.E. high-impedance when not powered.
But, since you didn't provide a schematic, no one can guess how your hardware is really connected.

You also didn't say what your project is doing, but is there a reason you have to use ancient technology (Serial)?

SteveMann:
This makes no sense:
"But apparently the arduino RX pin is connected to ground when not powered on,"

The Arduino RX pin should be floating, I.E. high-impedance when not powered.

Steve, probably not correct. You forgot the protection diodes. If both Vcc and 0V are the same (no power) then the 0V protection diode will prevent the pin going low and the Vcc protection diode will prevent it going high. It might be able to vary by about + or - 0V6, but that's all. It will be grounded for all practical purposes.

tompsonneke

Your description makes partial sense, but not completely. The ESP8266 Tx / RTS0 pin, GPIO15 should be pulled low for start up. I use something like a 3k3 resistor for this. Not sure if that's your problem, but if not I think it will be something along those lines.

SteveMann:
This makes no sense:
"But apparently the arduino RX pin is connected to ground when not powered on,"

The Arduino RX pin should be floating, I.E. high-impedance when not powered.
But, since you didn't provide a schematic, no one can guess how your hardware is really connected.

You also didn't say what your project is doing, but is there a reason you have to use ancient technology (Serial)?

Well, my thoughts exactly: "this makes no sense". But I tested it. When not powered on, the RX pin is connected to ground. I did include a schematic of the communication lines. They go directly to the TX/RX pins where I added the net labels. Any other schematics are not relevant here.
My arduino nano is driving a dmx chip and the esp8266 is used to connect to via a mqtt server to homebridge running on a raspberry pi. This info is however irrelevant, since I already have the board. And the communication from the ESP to the arduino is crucial. It sends the MQTT commands to the arduino.
And Serial connection is just a very easy (duplex with only 2 wires, easy coding ...) and reliable way to connect two boards on a PCB.

PerryBebbington:
tompsonneke

Your description makes partial sense, but not completely. The ESP8266 Tx / RTS0 pin, GPIO15 should be pulled low for start up. I use something like a 3k3 resistor for this. Not sure if that's your problem, but if not I think it will be something along those lines.

Apologies for the confusion. I use a NodeMCU board, so this is all taken care of. The board powers up just fine when I disconnect the arduino nano or turn the switch on the ESP Tx to nano Rx to "off". (so basically disconnecting the arduino RX pin from the ESP TX pin)

The standardESP8266 Tx pin (TXD0) is GPIO1. GPIO15 is the TXD2 pin and is a completely different pin than the one I'm using.

No apologies needed. I wasn't sure if you were describing a different problem, but I wanted to be sure it wasn't GPIO15 being high that was the problem. I nearly gave up on the ESP8266 until I discovered that because it was unreliable without the pull down resistor.