switch from bluetooth audio to jack and viceversa

Are you sure that with no jack inserted 3 & 6 and 2 & 5 are connected and not viceversa?

That is what the diagram in the data sheet is telling me. Is it telling you something different?