Internal pullup resistors are 30K to 50K so very weak pullups. the recommended pullup is 4.7K so external pullups are very likely necessary.
10K may be enough if the wires between the processor and I2C device are short ( a few centimeters) and the environment is not electrically noisy. Like I said, 4.7K is the recommended pullup. May need lower resistance in case of noise or longer wires.
Many of the common I2C modules have the pullup resistors built in already. Then you would not need more of them. If you use it for MCU <> MCU comms you could start with 2x 10K on one end and if it does not work reliably add another 2x 10K on the other end. Then you would have 5K in total, which is near perfect.
Both of those parts have the same internal pullups, however, the libraries used to make I2C work using a USI (there's no hardware I2C or SPI on those older tinies, just a USI which combines the two and is generally garbage) do not use the pullups; I'm not sure whether it's possible to do so with the constraints of the USI.
Even when you can use internal pullups, they're between 1/6th and 1/10th the strength required by the I2C spec, and while it may work with 1 device and short wires, it can easily fail depending on wire length and number of devices. This failing situation may be extremely sensitive to small changes and is hell to debug, and should be avoided. There is only one way to do this: use external pullups if they are not provided on the modules you're connecting it to.