Oh, ya, that'd do it.
Per the documentation ATTinyCore/ATtiny_x4.md at OldMaster---DO-NOT-SUBMIT-PRs-here-Use-2.0.0-dev · SpenceKonde/ATTinyCore · GitHub "Serial" uses a software serial implementation on the AIN0 and AIN1 pins (there's no hardware serial on the x4, so it has to be a software implementation.)
These pins are used to take advantage of the analog comparator interrupt, which is almost never used in arduino-land, in order to avoid using the PCINT or external INTn interupt vectors. The normal software serial libraries work on any pin, but use a PCINT vector, making them incompatible with other things that use PCINTs on a different pin. Naturally, if you're using those pins for Serial, you can't also use them for other purposes!