FTDI upload to standalone: COM in use when switching standalone boards

Before you switch, do you go thru USB Safely Remove Hardware thingy? Or just pull the cable out.
I just pull the cable, but I don't think I've connected more than 2 boards in a row. Usually just an FTDI adapter and the Atmel AVR ISP MKii adapter.