I don't think it's viable to provide for a specific connector(s) to be mounted directly onto the enclosure, there are too many options. I was suggesting a hole large enough to pass an RJ45 but leave where it goes up to the user.
Such a hole would probably also pass an IDC ribbon cable connector up to about 16 pins so maybe make it a bit larger to allow 26 or 34. Remember it doesn't have to go through square, you can sort of fold it back on the cable to make it smaller.
On the other side you have a similar problem because there are so many different LCDs, but it's worse because it has to look good unless you use bezels. Maybe try to find a hole size that matches several common displays (if there is such a thing as "common displays").
All the above could be handled with removable back/side/top plates as you say, there's just more tooling and inventory to hold and before long you don't have a box at all but a framework on which you hang plates

IMO finding the right enclosure is one of the hardest parts of a project and that may explain why most boards these days are just open, they leave that job up to the user.
_____
Rob