But i don't quite know if matlab arduino i/o package can use softwareserial library in arduino.
It can't. But, on a Mega with 4 hardware serial ports, why would you need/want to?
I hope i can wire another UART to other serialport on arduino. and connect the motorized stage to arduino through this new rs232.
You can.
I just don't know if i can send serial text to the new serialport through arduino i/o package in MATLAB.
You can't. You can send data to the Arduino on Serial, and let it do some work. It doesn't have to be an idiot-savant for you.
It can parse the data you send, and see that it is for another device, and pass it to that device using a SerialX instance.