RCA switchboard or parallel

I am not the first one on this forum, who wants to make a audio/composite switcher for multiple AV devices (e.g. dvd, ps3). But before I’m buying bus switches and other components, I was wondering about something.

Is it also possible to simply connect the video signals parallel to the output, and the same for the L/R audio. That way you don’t need any components other than wiring. I can imagine that, if you turn on multiple devices, I would get a distorted video and multiple audio sounds. But if I only turn on one device, there won’t be a problem right? Or is this harmful for the individual devices?