It's hard to comprehend your latency issue without seeing how you send data. Is it ASCII data or binary data? ASCII data is easier for humans to read, but it takes time to convert binary data to an ASCII representation, and the representation is typically larger than the binary data, meaning that it takes longer to send.
How are you measuring the latency? What marks the beginning of the latency period? What marks the end? I can't see how you can measure latency using an oscilloscope.