Yes.
No! Safety ground (the green dot in your PSU) connects to a metal chassis if your circuit is enclosed in one. If your project is housed in plastic, wood etc. you leave it disconnected.
Yes. But you weren't having a ground loop problem. It's also unlikely you'll have problems with ground loops; you might, but in my experience with circuits like yours, it's rare for them to crop up and actually cause problems. If you were building an audio amplifier, we'd be having a different discussion.
Look, I can't explain this to you, but just remember: in a typical circuit, all grounds connect together, except for the safety ground from your wall socket which is a special case.
Ok, maybe one more try: Floating ground - Wikipedia
Essentially the grounds of both your PSU's were floating. There was however no guarantee that they would float at the same level. I.e. the ground of PSU 1 might have been lifted 75V over the other and you would only notice by the components in your circuit dying. If you tie the grounds of PSU1 and PSU2 together, they are guaranteed to be at the same level (they're connected, after all). They may still float vis-a-vis absolute ground (i.e. planet earth, or rather, the potential of the water-logged soil below your house), but that's not a problem.