Manticorp:
and I just want to get a few things straight before I potentially damage a chip...
Just include 220 ? (or higher) resistors in series: Arduino ? resistor ? ATtiny85. You can never go wrong with a series resistor. 
I connect PB3 (which is transmitting) to the serial receiver and PB4 (which is receiving) to the serial transmitter.
I believe that is correct.
The Receive/Transmitter presumably can just be the TX/RX pins on an Arduino Uno for example?
Yes but you will have to remove the 328 processor from the circuit. You can either force it into reset by connecting a jumper from RESET to GND or you can remove the 328 processor from the board (with the power off).