An I2C version is also susceptible to parasitic powering over the I2C lines - maybe even through the pull-up resistors. The powered-down device may also pull low your I2C bus at all times, breaking communications to other devices.
1602 LCD displays may use a few mA even with the backlight off, sucking dry small batteries in weeks.