Serial port pins(programming port), consumes power when port removed from sketch

I need to achieve very low power consumption from the CPU. The serial port pins need to be connected to a serial port to minimize current consumption. If the external serial port are just disconnected, the current increases. There is about 80uA difference.

We need the serial port configured for programming, then disabled in the main program since is not needed there.

What could be getting configured in the CPU, that needs to be disabled in the main program?