Shift Register Libraries for LCD Displays

I really just wanted to know if this was the right way to do it. I heard that in some cases you can have multiple voltages in a system and you connect the grounds together. So basically the question is will the circuit short anything out. I will be using a seperate power supply for the arduino of course and the 9v can be replaced with a wall wort. I have been wanting to get a breadboard ps like that but for now i am using a voltage regulator and 2x 10uf caps with a .1 uf. Also, lets say I want to add a mosfet to power a high current device such as a solenoid. Would I also connect that ground to the circuit? Do you always connect all the grounds in these situations?

Thanks for the reply,
Anthony