Would you consider this a bad idea
Yes.
There is way too much processing going on for so little effect. Forget rounding because you are messing with the signal you will not perceive any diferance.
A simple shift to the right in your sample is all you need. So for a 64 level quantisation you just need to shift right four places:-
output = sample >> 4;