You need an input level that you can't alter with a signal from a channel. Something like this:-

What you have will not isolate the two outputs.
You need an input level that you can't alter with a signal from a channel. Something like this:-

What you have will not isolate the two outputs.