I2C with micro-USB power and multiple devices? (re: common ground)

My project involves an Arduino Leonardo using an external power adapter (ac-to-dc wall brick) and several Teensy 2.0 boards each connected to different PCs powered by the micro-USB from each PC. This means that every board has a different power source.

However, everything in my setup is using 5v but the power sources are different for each board that I'm connecting with I2C.

Do I just need to connect the I2C pins (SCL/SDA) and then connect a command GND between all devices?

EDIT: It looks like my other option is to separate the USB and 5v on the Teensy board and then share the incoming power from the Leonardo to each Teensy board that I'm connecting so they all use the same power source?

Thanks for any help