Why do this, you use pin mode only once in your code, or at least you should do.
What you call bloat is simply working what pin is being asked to change, and working out what register bit to change that maps to that pin. How are you going to cut that down? Have a look at the code for the pin mode function and see what is involved, then try and do better. I think you are worrying too much about this, and it is not a problem.
However, for a function that you use a lot, like digitalWrite, there might be a bit of fat to trim.
Look at this:-
For the sort if thing you need for that.