You need a 0.1uF ceramic cap between Vcc and Gnd on the tiny84 (you need this on almost all digital ICs, including all the AVR microcontrollers), as close to chip as possible (I'd put it over the top of the chip in this case). Without it, the part may not be stable (nearly-random hang/reset is possible).
Try doing some serial logging, either with the builtin software serial implementation (see ATTinyCore/ATtiny_x4.md at OldMaster---DO-NOT-SUBMIT-PRs-here-Use-2.0.0-dev · SpenceKonde/ATTinyCore · GitHub Serial section) or SoftwareSerial so you can see where it's hanging up.
Does the same code (with only pin number adjustments) work on the Uno? That's a good check to make sure that there isn't a problem hiding somewhere else.