Two ATTINY85s Together With I2C

I would be surprised if that's the issue. They have internal pullups.

I never use external reset pullups, unless there is a lot of EMF and long PCB traces on the reset pins.