"the arduino shouldn't need [I2C] pullups. The internal ones are enabled by default and are usually sufficient."
Barely. If you want decent rise times on the edges you need the 4.7K pullups. I missed the reference to 4.7K pulldowns. See what you expect to see sometimes.
If you looked at the signals with a 'scope you would really appreciate the difference.