Electronic equilevant of latching relay circuitry

In theory I2C will work so long as the pull-ups are to 3.3V (3.3V will read as high to a 5V Arduino),
but you must check the I2C driver isn't actively pulling up on the bus.

Without a common ground between IMU and Arduino it won't work at all...

And surely the base resistor goes on the base of the transistor after the diode(s), not in
parallel - you need to limit current however the transistor is operated.

I think you've fallen into the trap of using a single transistor to make a latching on-switch,
which can never work as transistors are inverting, so the powered down Arduino
is automatically going to be powering the transistor on via its protection diodes.