As I understand it, essentially when the board resets, and the proper information is sent through the rx and tx lines.
Correct, but the standard Arduino bootloader has short wait time after resetting it, before it exists the bootloader and starts the sketch. You have to be fast enough.
When using the auto-reset feature (serial DTR->Arduino RESET using a 100 nF capacitator http://arduino.cc/blog/?p=13), the wait-time should not be a problem. For BT devices this might not be what you want because the device is reset whenever you connect to it.
The Arduino BT uses a different bootloader which has a longer wait time. It also uses a different baudrate (115200) and initializes some parameters of the on-board WT11 bluetooth module, but it doesn't use auto-reset.
I have an external Bluetooth device which communicates just fine over a serial connection. It's baud rate is set 57600.
When I understand the code and the Makefile correct, you have to use 19200 baud instead for the diecimila. The ATmega328 bootloader seems to use 57600 baud.
Makefile (BAUD_RATE not set for diecimila)
diecimila: TARGET = diecimila
diecimila: CFLAGS += '-DMAX_TIME_COUNT=F_CPU>>4'
diecimila: AVR_FREQ = 16000000L
Makefile (BAUD_RATE set to 57600 for the ATmega328)
atmega328: TARGET = atmega328
atmega328: MCU_TARGET = atmega328p
atmega328: CFLAGS += '-DMAX_TIME_COUNT=F_CPU>>4'
atmega328: AVR_FREQ = 16000000L
atmega328: LDSECTION = --section-start=.text=0x7800
Code from arduino-0016\hardware\bootloaders\atmega\ATmegaBOOT_168.c
#define BAUD_RATE 19200
When you can't change the baud rate for the bluetooth chip, you have to change it in the boot loader or better use -DBAUD_RATE=57600.
Actually the baud rate between the computer and the bluetooth module doesn't matter, it can be completely different from the UART baudrate of the bluetooth module which is connected to the Arduino.
If I connect the RTS pin on the BT module to Arduino's reset pin, the board seems to react a little better
For the auto-reset you have to use DTR instead of RTS and don't connect the pin directly but via a capacitator, see above.