Some devices will not allow you to set an update rate that is too high for the baud rate. Try setting the GPS baud rate to 19200 or 38400:
gps.send_P( &gpsPort, F("PMTK314,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0") ); // RMC_GGA
delay( 100 );
gps.send_P( &gpsPort, F("PGCMD,33,0") ); // No antenna status messages needed
delay( 100 );
gps.send_P( &gps_port, F("PMTK251,19200") ); // set baud rate
gps_port.flush(); // wait for the command to go out
delay( 100 ); // wait for the GPS device to change speeds
gps_port.end(); // empty the input buffer, too
gps_port.begin( 19200 ); // use the new baud rate
gps.send_P( &gps_port, F("PMTK220,200") ); // set the GPS update interval
(Excerpt from this thread)