Are you using the circuit in Fig 4 of the HIH6130 datasheet?
Yes, I'm using figure 4 (left side) in the datasheet.
If so are you also removing power from the pullup resistors and the LED?
If I use any pullups, it doesn't really matter, for testing I usually skip them (so I don't need t use a breadboard), for production they are there. In this case the result is the same.
There is no LED. In the datasheet the LED or whatever is there is to show the use of the alarm triggers. I'm currently not using them (that is pins 5 an 6 on the HIH).
I don't remove power from the pullups, the are wired directly to 3.3V on the arduino.
Also when you say "remove power" are you driving pin 10 LOW or making it an input?
Off hand I can't see why this would matter but we don't know what's inside the sensor. It could be that driving VDD to 0v allows some sort of low-impedance path through to the SCL and SDA pins, If so that would kill the bus.
It's easy to test, either just remove the wire to VDD or change your code to set the power pin to an input.
EDIT: Yes I can see why it would matter, if there are ESD diodes inside the sensor you will have a short through them to 0v, that will clamp the signals to about .6v.