Problems with changing baud rate - device connected to Arduino Uno

The main problems:

Don't print too much. It interferes with receiving consecutive GPS characters. You are printing ~3240 characters for each sentence received from the GPS device. The Arduino is spending all its time waiting to print (read this).

Don't use SoftwareSerial. It disables interrupts for long periods of time, which will interfere with other parts of your program (or libraries). Read this for alternatives.

Don't use String™. Here are many gory details about why you shouldn't, and here's a good Arduino-centric summary.

Don't listen for the $PSIRF sentence, send it to change the baud rate:

    gpsPort.println( F("$PSRF100,1,9600,8,1,0*0D") ); // send the baudrate command

I think you probably need to do something like this:

//#define gpsPort Serial // BEST choice, but must rewire to pins 0 and 1

//#include <AltSoftSerial.h>
//AltSoftSerial gpsPort; // 2nd best choice, but must rewire to pins 8 and 9

#include <NeoSWSerial.h>
NeoSWSerial gpsPort( 11, 12 ); // 3rd best choice, works on pins 11 and 12

#include <NMEAGPS.h> // NeoGPS library
NMEAGPS gps;

void setup()
{
  Serial.begin( 4800 ); // Be sure the Serial Monitor window baud rate is also 4800

  gpsPort.begin( 4800 );                      // assume it's still @ 4800 (not needed if gpsPort *is* Serial)
  gpsPort.println( F("$PSRF100,1,9600,8,1,0*0D") ); // send the baudrate command
  gpsPort.flush();                            // and wait for it to be sent (1ms per character)

  delay( 200 );           // Let the GPS device start *sending* at new baudrate

  gpsPort.end();          // Throw away any received characters/garbage
  gpsPort.begin( 9600 );  // Start *receiving* at the new baudrate
}

void loop()
{
  if (gps.available( gpsPort )) {
    gps_fix fix = gps.read();

    Serial.print( F("Location: ") );
    if (fix.valid.location) {
      Serial.print( fix.latitude(), 6 );
      Serial.print( ',' );
      Serial.print( fix.longitude(), 6 );
    }

    Serial.print( F(", Altitude: ") );
    if (fix.valid.altitude)
      Serial.print( fix.altitude() );

    Serial.println();
  }
}

This is the NMEAsimple example from the NeoGPS library. If you want to try it, NeoGPS, NeoSWSerial and AltSoftSerial are available from the Arduino IDE Library Manager, under the menu Sketch -> Include Library -> Manage Libraries. Be sure to follow the Installation instructions. NeoGPS has an NMEAdiagnostic program to auto-detect what the GPS device is doing.

Cheers,
/dev