TinyGPS test_with_gps_device

I notice that only the tx led is flashing on the board and not the rx led, should that be the case?

The TX and RX LEDs are controlled by the USB-to-serial chip, not by the Arduino, and most certainly not by SoftwareSerial. So, yes, this is certainly possible.