Serial Port: is the same setting used for both programmer and bootloader?

It seems like the Serial Port setting is used both for talking to the programmer, and for talking to the board when using bootloader and/or serial monitor.
Typically, these are actually different ports, right?
I can't find a way to force a particular serial port in the programmer.txt file, either.
What's a good solution for this?

theres only one serial port on the arduino, the programmer is the bootloader. when the avr first comes up it runs the bootloader, which waits a second or two for stk500 commands, if it receives some it kicks into programmer mode, if not it moves on to your program.

I understand there is only one serial port on the Arduino. I'm talking about when using a separate programmer -- which there is a lot of support for in the IDE.
However, that support falls surprisingly short in this one place.