Using Power from two Arduinos

If I use the DC power supply to power this stuff should the ground be connected to the Arduino also or only the components I'm using to draw power from the power supply?