I power my board with external 5v, and have serial input as well.
The serial line is also high at 5v when no data is present.
If I then shut the 5v to switch the device off, I still see that SOME of serial line is leaking through the 328p chip, and pin 7 & 20 (VCC) are then seeing about 3.5v.
It's causing some issues.
You need to understand the data sheet.
If you look at the ATMel 328p Data Sheet Table 29-1. The Vih value:
It states the highest value that can be applied to a pin is Vcc+0.5 Volts. so if you are 'turning off' by setting the Vcc of the Arduino to 0V. the maximum Safe Voltage that can be applied to any PIN is 0.5V. Anything over this level is shunted through protection diodes to Vcc.
So, effectively you are powering up the 'powered down' CPU by forcing power through an pin.
Here is the schematic for AVR 328p pins:
When the processor is powered down, all pins (inputs and outputs) must be powered down.
If the Serial port is still at 5V you are going to have to install a tri-state buffer between the CPU and external connections.
something like a 74LS126 with its control input 'G' connected to Vcc, the 'Y' connected to RX, 'A' from the input source. But this 74ls126 would still need to be powered while the CPU is off.