Feasibility of an ethernet-based matrix switcher?

Forgive me if this has been asked and answered a hundred times. My searches didn't turn up anything specific to my goal, perhaps because of the nature of the terms I have to search for.

The background of my question is that I work for a non-profit with an existing but very old standard-definition-based Crestron system that coordinates a number of inputs and outputs in three different meeting rooms. To upgrade the system to modern standards is way above what we can afford to budget at the present time.

That being the case and in an effort to get away from proprietary systems, I was looking at doing something with video over ethernet using HDMI extenders. The biggest difficulty I'm coming up against is switching the various signals to the desired locations. Obviously there are matrix switches out there for HDbaseT, however, the ones I've seen, besides being a little pricey up front take the HDMI-ins right at the switch and not from the remote locations as our current setup is.

My thought process was that if an arbitrary number of ethernet ports could be wired together in such a way that they would act as a glorified patch panel that could be controlled by a device, such as an Arduino, that this would accomplish a significant portion of my goals in this experiment. If, additionally, the signal could be amplified such that an input port could be routed to multiple output ports, that would be great as well to accomplish the desire to be able to overflow from one room to another.

There are other obvious challenges in this concept but I figure if this weren't an issue that could be overcome, there wouldn't be much point in digging further. I would envision controlling the system from a web-based interface with a separate web server and interacting with the various components over regular ethernet connections.

Thanks for any guidance anyone can offer.

I'm not a network expert, but I could imagine that each source should become a server, to which all other clients can attach freely. Data transfer would occur in data packets, not HDMI signals. The clients could be RasPi's with graphics hardware.