Then try with hardware serial (without using usb-serial for serialprints).
Common ground along the lines could help.
And the rs485 line might need termination resistors. Not all modules have it on board.
Is the corrupted byte always the same one? First one?