I don't think you are measuring the delay from the libraries at all - especially not with the two versions of TinyGPS. At 4800 baud it takes 2.08ms to transmit one character. The receiving side can't start to transmit a received character until it sees the stop bit which therefore takes about 2.08ms between the start of the sender's transmission and the start of the receiver's transmission. All you're measuring is the 4800 baud transmission rate.
Your code for the NMEA library actually calls the library and prints the decoded result which causes a longer delay (dunno whether you've measured it correctly though). The code for both TinyGPS sketches simply relays whatever character is received from the GPS and your oscilloscope images confirm this.
Pete